[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:29 dist-test-slave-l9ww krb5kdc[1221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:29 dist-test-slave-l9ww krb5kdc[1221](info): set up 2 sockets
Sep 25 12:21:29 dist-test-slave-l9ww krb5kdc[1221](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 12:21:30.029306 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:30.151083 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43791
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:43791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:30.232084 1234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:30.232306 1234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:30.232329 1234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:30.233525 1234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:30.233572 1234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:30.233587 1234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:30.233598 1234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:30.233611 1234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:30.235155 1234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:43791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:30.235433 1234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:30.235775 1234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:30.241196 1242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.241941 1240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.242555 1234 server_base.cc:1034] running on GCE node
W20260925 12:21:30.242694 1239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.242959 1234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:30.243292 1234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:30.244454 1234 hybrid_clock.cc:648] HybridClock initialized: now 1790338890244414 us; error 51 us; skew 500 ppm
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:30.254578 1234 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:30.255254 1234 webserver.cc:460] Webserver started at http://127.1.47.190:37011/ using document root <none> and password file <none>
I20260925 12:21:30.255474 1234 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:30.255520 1234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:30.255622 1234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:30.256889 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "fa23c6f995254db3a770613f9aee01a0"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "41e5addff70b57a9bc20a7946afb418f"
server_key_iv: "0267f4a6d173598c5493cbe780dbe015"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.257237 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "fa23c6f995254db3a770613f9aee01a0"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "41e5addff70b57a9bc20a7946afb418f"
server_key_iv: "0267f4a6d173598c5493cbe780dbe015"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.258849 1234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:30.259690 1249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.259985 1234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:30.260098 1234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "fa23c6f995254db3a770613f9aee01a0"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "41e5addff70b57a9bc20a7946afb418f"
server_key_iv: "0267f4a6d173598c5493cbe780dbe015"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.260190 1234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:30.280742 1234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:30.283003 1234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:30.283200 1234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:30.287598 1234 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:43791
I20260925 12:21:30.287637 1301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:43791 every 8 connection(s)
I20260925 12:21:30.288046 1234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:30.289021 1302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:30.291385 1302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0: Bootstrap starting.
I20260925 12:21:30.292060 1302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:30.292315 1302 log.cc:824] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:30.293079 1302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0: No bootstrap required, opened a new log
I20260925 12:21:30.294742 1302 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } }
I20260925 12:21:30.294888 1302 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:30.294915 1302 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa23c6f995254db3a770613f9aee01a0, State: Initialized, Role: FOLLOWER
I20260925 12:21:30.295025 1302 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } }
I20260925 12:21:30.295110 1302 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:30.295149 1302 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:30.295228 1302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:30.295814 1302 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } }
I20260925 12:21:30.295971 1302 leader_election.cc:302] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa23c6f995254db3a770613f9aee01a0; no voters:
I20260925 12:21:30.296155 1302 leader_election.cc:288] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:30.296206 1305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:30.296387 1305 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [term 1 LEADER]: Becoming Leader. State: Replica: fa23c6f995254db3a770613f9aee01a0, State: Running, Role: LEADER
I20260925 12:21:30.296442 1302 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:30.296504 1305 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } }
I20260925 12:21:30.296804 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1234
I20260925 12:21:30.296937 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:30.297101 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa23c6f995254db3a770613f9aee01a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } } }
I20260925 12:21:30.297243 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:30.297534 1315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:30.297816 1214 external_mini_cluster.cc:1428] Setting key 6bcf87f5dd217d83960a8dbe40d16ba5
I20260925 12:21:30.298027 1307 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa23c6f995254db3a770613f9aee01a0. Latest consensus state: current_term: 1 leader_uuid: "fa23c6f995254db3a770613f9aee01a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa23c6f995254db3a770613f9aee01a0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43791 } } }
I20260925 12:21:30.298110 1307 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:30.298362 1315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:30.300091 1315 catalog_manager.cc:1347] Generated new cluster ID: cd565fb4d0744681b0a4760f333b8358
I20260925 12:21:30.300143 1315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:30.313012 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.298853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.1.47.190:43791)
0925 12:21:30.299043 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.299052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.299407 (+ 355us) server_negotiation.cc:407] Connection header received
0925 12:21:30.301434 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.301439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.301492 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.301643 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.303175 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.304188 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.305232 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.305636 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.305814 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.309332 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.309336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.309337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.309356 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.311072 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.311535 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.311538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.311538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.311631 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.311926 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.311927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.311927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.312178 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.312343 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.312622 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.312754 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260925 12:21:30.322584 1315 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:30.323323 1315 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:30.334846 1315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa23c6f995254db3a770613f9aee01a0: Generated new TSK 0
I20260925 12:21:30.335078 1315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:30.367794 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:30.456266 1328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:30.456522 1328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:30.456565 1328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:30.457775 1328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:30.457851 1328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:30.457899 1328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:30.459528 1328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:30.459827 1328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:30.460168 1328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:30.465636 1336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.465765 1333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.465971 1334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.466074 1328 server_base.cc:1034] running on GCE node
I20260925 12:21:30.466265 1328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:30.466467 1328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:30.467617 1328 hybrid_clock.cc:648] HybridClock initialized: now 1790338890467601 us; error 31 us; skew 500 ppm
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:30.470327 1328 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:30.470912 1328 webserver.cc:460] Webserver started at http://127.1.47.129:46627/ using document root <none> and password file <none>
I20260925 12:21:30.471158 1328 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:30.471228 1328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:30.471385 1328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:30.472561 1328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "e87f3c53204741cf99c2ed30a5b8ea72"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "7c6ed1224f2ceb4515dd4e9af941f1ce"
server_key_iv: "18fe6ae1cfb096b548a8bab55630dffe"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.472977 1328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "e87f3c53204741cf99c2ed30a5b8ea72"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "7c6ed1224f2ceb4515dd4e9af941f1ce"
server_key_iv: "18fe6ae1cfb096b548a8bab55630dffe"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.474433 1328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:30.475281 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.475507 1328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:30.475615 1328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "e87f3c53204741cf99c2ed30a5b8ea72"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "7c6ed1224f2ceb4515dd4e9af941f1ce"
server_key_iv: "18fe6ae1cfb096b548a8bab55630dffe"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.475693 1328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:30.489445 1328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:30.491696 1328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:30.491991 1328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:30.492240 1328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:30.492578 1328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:30.492636 1328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.492676 1328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:30.492702 1328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.500371 1328 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:46467
I20260925 12:21:30.500420 1456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:46467 every 8 connection(s)
I20260925 12:21:30.500795 1328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:30.504004 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1328
I20260925 12:21:30.504079 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:30.504248 1214 external_mini_cluster.cc:1428] Setting key 5644fb086506c16f3ff764b0d36bdbe4
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:30.511788 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.501627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33081 (local address 127.1.47.190:43791)
0925 12:21:30.501738 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.501740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.501932 (+ 192us) server_negotiation.cc:407] Connection header received
0925 12:21:30.502478 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.502480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.502494 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.502552 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.503530 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.504198 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.505408 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.505669 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.505767 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.508504 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.508506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.508506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.508514 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.510124 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.510714 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.510715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.510715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.510775 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.511090 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.511091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.511091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.511201 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.511274 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.511636 (+ 362us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.511718 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:21:30.512122 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.501584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43791 (local address 127.1.47.129:33081)
0925 12:21:30.501800 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:30.501839 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:30.502354 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:30.502631 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:30.502635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:30.503217 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:30.503417 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.503422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.504291 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.504293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.505313 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.505316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.505792 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.505793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.505977 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.506558 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:30.506568 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:30.508374 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:30.510221 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.510228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.510231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.510597 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.510855 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.510856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.510856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.511016 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.511272 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:30.511279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:30.511454 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 12:21:30.511591 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 12:21:30.511793 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
I20260925 12:21:30.512766 1457 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43791
I20260925 12:21:30.512887 1457 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:30.513070 1457 heartbeater.cc:511] Master 127.1.47.190:43791 requested a full tablet report, sending...
I20260925 12:21:30.513717 1266 ts_manager.cc:194] Registered new tserver with Master: e87f3c53204741cf99c2ed30a5b8ea72 (127.1.47.129:46467)
I20260925 12:21:30.514305 1266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:33081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:30.531039 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:30.615368 1462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:30.615605 1462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:30.615630 1462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:30.616897 1462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:30.616947 1462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:30.616976 1462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:30.618460 1462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:30.618718 1462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:30.618959 1462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:30.624173 1470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.624222 1467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.624334 1462 server_base.cc:1034] running on GCE node
W20260925 12:21:30.624176 1468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.624672 1462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:30.624933 1462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:30.626127 1462 hybrid_clock.cc:648] HybridClock initialized: now 1790338890626118 us; error 47 us; skew 500 ppm
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:30.628818 1462 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:30.629244 1462 webserver.cc:460] Webserver started at http://127.1.47.130:33113/ using document root <none> and password file <none>
I20260925 12:21:30.629479 1462 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:30.629549 1462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:30.629700 1462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:30.630801 1462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "a52fab583cb84cc08780688c1ff8612d"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "1610e5ce52c290bf3b02be7017988509"
server_key_iv: "8b19f751dc43ca309187174cb40430ab"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.631177 1462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "a52fab583cb84cc08780688c1ff8612d"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "1610e5ce52c290bf3b02be7017988509"
server_key_iv: "8b19f751dc43ca309187174cb40430ab"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.632992 1462 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 12:21:30.633854 1477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.634052 1462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:30.634171 1462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "a52fab583cb84cc08780688c1ff8612d"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "1610e5ce52c290bf3b02be7017988509"
server_key_iv: "8b19f751dc43ca309187174cb40430ab"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.634253 1462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:30.651185 1462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:30.653502 1462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:30.653684 1462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:30.653936 1462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:30.654659 1462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:30.654721 1462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.654765 1462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:30.654790 1462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.661403 1462 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:37733
I20260925 12:21:30.661482 1590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:37733 every 8 connection(s)
I20260925 12:21:30.661877 1462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:30.666438 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1462
I20260925 12:21:30.666534 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:30.666702 1214 external_mini_cluster.cc:1428] Setting key 3c3acfe478e8ba951128945a3db2af23
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:30.672776 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.662628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43791 (local address 127.1.47.130:55045)
0925 12:21:30.662813 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:30.662851 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:30.663393 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:30.663662 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:30.663666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:30.664349 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:30.664550 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.664556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.665271 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.665272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.666076 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.666079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.666629 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.666631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.666766 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.667406 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:30.667416 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:30.669405 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:30.671285 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.671291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.671293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.671528 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.671750 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.671751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.671751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.671921 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.672238 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:30.672243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:30.672389 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 12:21:30.672476 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 12:21:30.672606 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260925 12:21:30.672782 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.662661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:55045 (local address 127.1.47.190:43791)
0925 12:21:30.662749 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.662751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.662935 (+ 184us) server_negotiation.cc:407] Connection header received
0925 12:21:30.663523 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.663524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.663538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.663587 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.664665 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.665174 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.666177 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.666512 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.666589 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.669507 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.669509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.669509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.669517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.671206 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.671607 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.671609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.671609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.671678 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.671997 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.671998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.671998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.672165 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.672245 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.672545 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.672643 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 12:21:30.673237 1591 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43791
I20260925 12:21:30.673367 1591 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:30.673566 1591 heartbeater.cc:511] Master 127.1.47.190:43791 requested a full tablet report, sending...
I20260925 12:21:30.673974 1266 ts_manager.cc:194] Registered new tserver with Master: a52fab583cb84cc08780688c1ff8612d (127.1.47.130:37733)
I20260925 12:21:30.674523 1266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:55045
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:30.695835 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:30.779800 1596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:30.780092 1596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:30.780136 1596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:30.781368 1596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:30.781440 1596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:30.781486 1596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:30.783033 1596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:30.783313 1596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:30.783557 1596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:30.788872 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.788774 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:30.788959 1602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:30.789260 1596 server_base.cc:1034] running on GCE node
I20260925 12:21:30.789516 1596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:30.789803 1596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:30.790979 1596 hybrid_clock.cc:648] HybridClock initialized: now 1790338890790963 us; error 46 us; skew 500 ppm
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:30.793496 1596 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:30.794021 1596 webserver.cc:460] Webserver started at http://127.1.47.131:34073/ using document root <none> and password file <none>
I20260925 12:21:30.794217 1596 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:30.794260 1596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:30.794360 1596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:30.795392 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "ed964c9e95df4d109a7561e0b8d7a8f3"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "566d82c0d2e3b4a145f7f76f8a868014"
server_key_iv: "d0a984f11bc9be6dbe422a95347b33a8"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.795699 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "ed964c9e95df4d109a7561e0b8d7a8f3"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "566d82c0d2e3b4a145f7f76f8a868014"
server_key_iv: "d0a984f11bc9be6dbe422a95347b33a8"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.797268 1596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:30.798056 1611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.798218 1596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:30.798301 1596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "ed964c9e95df4d109a7561e0b8d7a8f3"
format_stamp: "Formatted at 2026-09-25 12:21:30 on dist-test-slave-l9ww"
server_key: "566d82c0d2e3b4a145f7f76f8a868014"
server_key_iv: "d0a984f11bc9be6dbe422a95347b33a8"
server_key_version: "encryptionkey@0"
I20260925 12:21:30.798357 1596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:30.823325 1596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:30.825848 1596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:30.825974 1596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:30.826174 1596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:30.826504 1596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:30.826540 1596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.826565 1596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:30.826576 1596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:30.833173 1596 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:44293
I20260925 12:21:30.833249 1724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:44293 every 8 connection(s)
I20260925 12:21:30.833580 1596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:30.840934 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1596
I20260925 12:21:30.841004 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:30.841164 1214 external_mini_cluster.cc:1428] Setting key 7c47a8eaf8c99e8b6fdddd45a0acaa3e
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:30.844731 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.834383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54461 (local address 127.1.47.190:43791)
0925 12:21:30.834450 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.834452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.834824 (+ 372us) server_negotiation.cc:407] Connection header received
0925 12:21:30.835453 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.835455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.835468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.835524 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.836678 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.837298 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.838893 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.839265 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.839341 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.841992 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.841994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.841994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.842002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.843374 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.843804 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.843805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.843805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.843911 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.844167 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.844168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.844168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.844281 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.844335 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.844583 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.844651 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:21:30.844916 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.834394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43791 (local address 127.1.47.131:54461)
0925 12:21:30.834656 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:30.834724 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:30.835323 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:30.835615 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:30.835620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:30.836345 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:30.836558 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.836564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.837393 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.837394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.838798 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:30.838801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.839350 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.839352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:30.839496 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.840094 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:30.840103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:30.841902 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:30.843457 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.843462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.843464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.843707 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.843998 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:30.843998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:30.843999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:30.844100 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:30.844343 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:30.844347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:30.844469 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 12:21:30.844555 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:21:30.844676 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260925 12:21:30.845398 1725 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43791
I20260925 12:21:30.845480 1725 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:30.845623 1725 heartbeater.cc:511] Master 127.1.47.190:43791 requested a full tablet report, sending...
I20260925 12:21:30.845957 1266 ts_manager.cc:194] Registered new tserver with Master: ed964c9e95df4d109a7561e0b8d7a8f3 (127.1.47.131:44293)
I20260925 12:21:30.846307 1266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:54461
I20260925 12:21:30.853207 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:30.876986 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.868989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.1.47.129:46467)
0925 12:21:30.869177 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.869182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.869193 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:30.869219 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.869221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.869234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.869332 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.869865 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.870602 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.871337 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.871660 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.871713 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.874022 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.874034 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.874034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.874043 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.875746 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.876129 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.876133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.876136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.876195 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.876392 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.876393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.876394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.876542 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.876725 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.876770 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.876854 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":127,"threads_started":1}
W20260925 12:21:30.877370 1371 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34140
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:30.900404 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:30.892884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.1.47.129:46467)
0925 12:21:30.892975 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:30.892977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:30.892992 (+ 15us) server_negotiation.cc:407] Connection header received
0925 12:21:30.893072 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:30.893073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:30.893083 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:30.893134 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:30.893729 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.894218 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.894992 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:30.895196 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:30.895254 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:30.897685 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:30.897686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:30.897686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:30.897693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:30.899195 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.899561 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.899562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.899562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.899621 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:30.899838 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:30.899838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:30.899839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:30.900020 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:30.900123 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:30.900251 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 12:21:30.900314 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 12:21:30.901115 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1328
I20260925 12:21:30.907375 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1462
I20260925 12:21:30.911980 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1596
I20260925 12:21:30.916230 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1234
2026-09-25T12:21:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1949 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1739](info): set up 2 sockets
Sep 25 12:21:30 dist-test-slave-l9ww krb5kdc[1739](info): commencing operation
krb5kdc: starting...
W20260925 12:21:31.975999 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:32.097045 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35325
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:32.178884 1752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:32.179222 1752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:32.179265 1752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:32.180541 1752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:32.180610 1752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:32.180635 1752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:32.180662 1752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:32.180693 1752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:32.182193 1752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:35325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:32.182487 1752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:32.182729 1752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:32.187500 1760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.187500 1758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.187628 1757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.187906 1752 server_base.cc:1034] running on GCE node
I20260925 12:21:32.188171 1752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:32.188488 1752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:32.189687 1752 hybrid_clock.cc:648] HybridClock initialized: now 1790338892189674 us; error 40 us; skew 500 ppm
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:32.192409 1752 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:32.192917 1752 webserver.cc:460] Webserver started at http://127.1.47.190:41525/ using document root <none> and password file <none>
I20260925 12:21:32.193166 1752 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:32.193235 1752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:32.193383 1752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:32.194464 1752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "161ea4d942bb444eae8d47680748bff9"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "ea66f9bee8478e05874509278d002eda"
server_key_iv: "d24a6287a0470694e041bb2aca39e29c"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.194834 1752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "161ea4d942bb444eae8d47680748bff9"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "ea66f9bee8478e05874509278d002eda"
server_key_iv: "d24a6287a0470694e041bb2aca39e29c"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.196398 1752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:32.197153 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.197357 1752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 12:21:32.197463 1752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "161ea4d942bb444eae8d47680748bff9"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "ea66f9bee8478e05874509278d002eda"
server_key_iv: "d24a6287a0470694e041bb2aca39e29c"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.197536 1752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:32.216063 1752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:32.218308 1752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:32.218508 1752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:32.223089 1752 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:35325
I20260925 12:21:32.223124 1819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:35325 every 8 connection(s)
I20260925 12:21:32.223564 1752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:32.224601 1820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:32.226945 1820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9: Bootstrap starting.
I20260925 12:21:32.227558 1820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:32.227937 1820 log.cc:824] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:32.228663 1820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9: No bootstrap required, opened a new log
I20260925 12:21:32.230275 1820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } }
I20260925 12:21:32.230419 1820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:32.230443 1820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161ea4d942bb444eae8d47680748bff9, State: Initialized, Role: FOLLOWER
I20260925 12:21:32.230605 1820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } }
I20260925 12:21:32.230693 1820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:32.230719 1820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:32.230788 1820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:32.231312 1820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } }
I20260925 12:21:32.231428 1820 leader_election.cc:302] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161ea4d942bb444eae8d47680748bff9; no voters:
I20260925 12:21:32.231622 1820 leader_election.cc:288] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:32.231715 1823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:32.231940 1820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:32.231988 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1752
I20260925 12:21:32.232079 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:32.232261 1214 external_mini_cluster.cc:1428] Setting key c04cd394c26da42fad6f230da72a04f0
I20260925 12:21:32.232915 1823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [term 1 LEADER]: Becoming Leader. State: Replica: 161ea4d942bb444eae8d47680748bff9, State: Running, Role: LEADER
I20260925 12:21:32.233049 1823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } }
I20260925 12:21:32.233421 1823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161ea4d942bb444eae8d47680748bff9. Latest consensus state: current_term: 1 leader_uuid: "161ea4d942bb444eae8d47680748bff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } } }
I20260925 12:21:32.233459 1824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161ea4d942bb444eae8d47680748bff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ea4d942bb444eae8d47680748bff9" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35325 } } }
I20260925 12:21:32.233541 1823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:32.233584 1824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:32.234243 1840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:32.234714 1840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:32.236554 1840 catalog_manager.cc:1347] Generated new cluster ID: 469715e0990341f581fe484431a5eda4
I20260925 12:21:32.236608 1840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:32.245764 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.233065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.1.47.190:35325)
0925 12:21:32.233897 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.233908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.233941 (+ 33us) server_negotiation.cc:407] Connection header received
0925 12:21:32.234054 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.234060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.234114 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.234278 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.235060 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.236301 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.237434 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.238088 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.238202 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.241260 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.241265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.241266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.241281 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.243504 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.243985 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.243989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.243990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.244124 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.244337 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.244337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.244338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.244677 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.244843 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.245366 (+ 523us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.245517 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":64,"threads_started":1}
I20260925 12:21:32.266770 1840 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:32.267333 1840 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:32.273286 1840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 161ea4d942bb444eae8d47680748bff9: Generated new TSK 0
I20260925 12:21:32.273543 1840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:32.311213 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:32.406128 1846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:32.406368 1846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:32.406394 1846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:32.408015 1846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:32.408071 1846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:32.408128 1846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:32.409971 1846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:32.410270 1846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:32.410498 1846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:32.416518 1852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.416522 1851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.416680 1846 server_base.cc:1034] running on GCE node
W20260925 12:21:32.416525 1854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.416934 1846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:32.417245 1846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:32.418449 1846 hybrid_clock.cc:648] HybridClock initialized: now 1790338892418423 us; error 57 us; skew 500 ppm
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:32.422643 1846 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:32.423383 1846 webserver.cc:460] Webserver started at http://127.1.47.129:43623/ using document root <none> and password file <none>
I20260925 12:21:32.423688 1846 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:32.423758 1846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:32.423944 1846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:32.425005 1846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "ff367707540c448aa7f2d99cdaab549e"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cd7b0af6bcd1ce172a70c8c09e925e05"
server_key_iv: "7e95622bfb1fdf574a3def25373a3cab"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.425473 1846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "ff367707540c448aa7f2d99cdaab549e"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cd7b0af6bcd1ce172a70c8c09e925e05"
server_key_iv: "7e95622bfb1fdf574a3def25373a3cab"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.427170 1846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:32.428105 1861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.428263 1846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:32.428351 1846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "ff367707540c448aa7f2d99cdaab549e"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cd7b0af6bcd1ce172a70c8c09e925e05"
server_key_iv: "7e95622bfb1fdf574a3def25373a3cab"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.428418 1846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:32.450834 1846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:32.453390 1846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:32.453516 1846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:32.453725 1846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:32.454069 1846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:32.454111 1846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.454135 1846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:32.454149 1846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.460915 1846 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:45963
I20260925 12:21:32.460991 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:45963 every 8 connection(s)
I20260925 12:21:32.461349 1846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:32.467116 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1846
I20260925 12:21:32.467254 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:32.467473 1214 external_mini_cluster.cc:1428] Setting key e75120dc96fbe43d005ae2eab4b8742f
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:32.473865 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.462253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35325 (local address 127.1.47.129:59789)
0925 12:21:32.462445 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.462494 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.463516 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.463818 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.463822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.464199 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:32.464418 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.464423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.465196 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.465197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.466131 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.466133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.466536 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.466537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.466672 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.467438 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:32.467451 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:32.469526 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:32.471745 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.471754 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.471758 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.472184 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.472402 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.472403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.472403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.472546 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.473118 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:32.473125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:32.473302 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.473412 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.473606 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260925 12:21:32.474110 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.462313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:59789 (local address 127.1.47.190:35325)
0925 12:21:32.462382 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.462384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.462593 (+ 209us) server_negotiation.cc:407] Connection header received
0925 12:21:32.463634 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.463635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.463645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.463713 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.464537 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.465094 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.466244 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.466445 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.466510 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.469663 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.469665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.469665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.469674 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.471642 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.472252 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.472254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.472254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.472333 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.472847 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.472848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.472848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.473036 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.473096 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.473918 (+ 822us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.474032 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:32.474823 1975 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35325
I20260925 12:21:32.474953 1975 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:32.475144 1975 heartbeater.cc:511] Master 127.1.47.190:35325 requested a full tablet report, sending...
I20260925 12:21:32.475689 1784 ts_manager.cc:194] Registered new tserver with Master: ff367707540c448aa7f2d99cdaab549e (127.1.47.129:45963)
I20260925 12:21:32.476678 1784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:59789
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:32.498768 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:32.581447 1980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:32.581673 1980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:32.581700 1980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:32.583081 1980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:32.583145 1980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:32.583194 1980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:32.585882 1980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:32.586184 1980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:32.586478 1980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:32.591722 1986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.591737 1988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.592087 1985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.592161 1980 server_base.cc:1034] running on GCE node
I20260925 12:21:32.592371 1980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:32.592584 1980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:32.593797 1980 hybrid_clock.cc:648] HybridClock initialized: now 1790338892593782 us; error 32 us; skew 500 ppm
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:32.596524 1980 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:32.596927 1980 webserver.cc:460] Webserver started at http://127.1.47.130:37367/ using document root <none> and password file <none>
I20260925 12:21:32.597182 1980 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:32.597229 1980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:32.597368 1980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:32.598444 1980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "f870b120301a45eba1ac7480cb3880a3"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "9689002f7aa5f84872872c6ab6eacb5f"
server_key_iv: "27c3a2c9d795b8319338cd27c20a1d8f"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.598832 1980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "f870b120301a45eba1ac7480cb3880a3"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "9689002f7aa5f84872872c6ab6eacb5f"
server_key_iv: "27c3a2c9d795b8319338cd27c20a1d8f"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.600248 1980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:32.601020 1995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.601203 1980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:32.601291 1980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "f870b120301a45eba1ac7480cb3880a3"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "9689002f7aa5f84872872c6ab6eacb5f"
server_key_iv: "27c3a2c9d795b8319338cd27c20a1d8f"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.601349 1980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:32.627272 1980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:32.629632 1980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:32.629761 1980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:32.629961 1980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:32.630326 1980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:32.630366 1980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.630391 1980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:32.630403 1980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.636823 1980 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:41379
I20260925 12:21:32.636902 2108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:41379 every 8 connection(s)
I20260925 12:21:32.637238 1980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:32.644202 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 1980
I20260925 12:21:32.644333 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:32.644513 1214 external_mini_cluster.cc:1428] Setting key bca32a05508fd26258ad06409cc0e175
I20260925 12:21:32.647579 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.638157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:34347 (local address 127.1.47.190:35325)
0925 12:21:32.638227 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.638229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.638346 (+ 117us) server_negotiation.cc:407] Connection header received
0925 12:21:32.639314 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.639315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.639327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.639389 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.640040 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.640548 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.641655 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.641853 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.641908 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.644269 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.644271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.644271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.644280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.646083 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.646509 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.646510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.646510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.646574 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.646905 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.646906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.646906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.647028 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.647109 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.647404 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.647486 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:21:32.647801 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.638057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35325 (local address 127.1.47.130:34347)
0925 12:21:32.638218 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.638261 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.639194 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.639455 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.639459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.639724 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:32.639954 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.639958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.640666 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.640667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.641568 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.641570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.641956 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.641957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.642118 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.642623 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:32.642632 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:32.644167 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:32.646176 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.646182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.646184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.646429 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.646662 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.646663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.646664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.646828 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.647107 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:32.647115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:32.647265 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.647373 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.647545 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260925 12:21:32.648288 2109 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35325
I20260925 12:21:32.648409 2109 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:32.648598 2109 heartbeater.cc:511] Master 127.1.47.190:35325 requested a full tablet report, sending...
I20260925 12:21:32.649078 1784 ts_manager.cc:194] Registered new tserver with Master: f870b120301a45eba1ac7480cb3880a3 (127.1.47.130:41379)
I20260925 12:21:32.649466 1784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:34347
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:32.671674 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:32.759254 2114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:32.759478 2114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:32.759503 2114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:32.760746 2114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:32.760800 2114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:32.760833 2114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:32.762298 2114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:32.762493 2114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:32.762672 2114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:32.767598 2122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:32.767593 2120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.767840 2114 server_base.cc:1034] running on GCE node
W20260925 12:21:32.767596 2119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:32.768117 2114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:32.768392 2114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:32.769552 2114 hybrid_clock.cc:648] HybridClock initialized: now 1790338892769532 us; error 37 us; skew 500 ppm
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:32.772156 2114 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:32.772642 2114 webserver.cc:460] Webserver started at http://127.1.47.131:45101/ using document root <none> and password file <none>
I20260925 12:21:32.772874 2114 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:32.772956 2114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:32.773132 2114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:32.774655 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "fe56be4fd4834ef5a955fbcecd61a652"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cdc30ccd1fedd375f699b45ae2ca1612"
server_key_iv: "0dc6d73012ca8b5b2a958e41091b62a4"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.775096 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "fe56be4fd4834ef5a955fbcecd61a652"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cdc30ccd1fedd375f699b45ae2ca1612"
server_key_iv: "0dc6d73012ca8b5b2a958e41091b62a4"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.776711 2114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260925 12:21:32.777630 2129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.777819 2114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:32.777926 2114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "fe56be4fd4834ef5a955fbcecd61a652"
format_stamp: "Formatted at 2026-09-25 12:21:32 on dist-test-slave-l9ww"
server_key: "cdc30ccd1fedd375f699b45ae2ca1612"
server_key_iv: "0dc6d73012ca8b5b2a958e41091b62a4"
server_key_version: "encryptionkey@0"
I20260925 12:21:32.777985 2114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:32.786722 2114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:32.788808 2114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:32.788924 2114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:32.789109 2114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:32.789428 2114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:32.789463 2114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.789508 2114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:32.789533 2114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:32.795284 2114 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:36827
I20260925 12:21:32.795356 2242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:36827 every 8 connection(s)
I20260925 12:21:32.795707 2114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:32.797263 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2114
I20260925 12:21:32.797343 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:32.797508 1214 external_mini_cluster.cc:1428] Setting key e7e926e735c7f95fdcb39e70c8e03c38
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:32.806172 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.796661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:60969 (local address 127.1.47.190:35325)
0925 12:21:32.796739 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.796742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.796989 (+ 247us) server_negotiation.cc:407] Connection header received
0925 12:21:32.798157 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.798158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.798171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.798242 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.798888 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.799423 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.800762 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.800949 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.801015 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.803333 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.803335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.803335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.803342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.804830 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.805327 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.805328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.805328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.805394 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.805629 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.805630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.805630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.805744 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.805794 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.806036 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.806102 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:32.806367 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.796653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35325 (local address 127.1.47.131:60969)
0925 12:21:32.796842 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.796892 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.798029 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.798330 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.798334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.798622 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:32.798813 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.798817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.799530 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.799531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.800678 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.800680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.801066 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.801067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.801186 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.801774 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:32.801783 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:32.803222 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:32.804929 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.804935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.804937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.805179 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.805455 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:32.805455 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:32.805455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:32.805573 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:32.805798 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:32.805802 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:32.805925 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.806008 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.806146 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260925 12:21:32.806846 2243 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35325
I20260925 12:21:32.806942 2243 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:32.807263 2243 heartbeater.cc:511] Master 127.1.47.190:35325 requested a full tablet report, sending...
I20260925 12:21:32.807821 1784 ts_manager.cc:194] Registered new tserver with Master: fe56be4fd4834ef5a955fbcecd61a652 (127.1.47.131:36827)
I20260925 12:21:32.808274 1784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:60969
I20260925 12:21:32.809211 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:21:32.816762 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.810611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35932 (local address 127.1.47.190:35325)
0925 12:21:32.810689 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.810691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.810787 (+ 96us) server_negotiation.cc:407] Connection header received
0925 12:21:32.810857 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.810857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.810868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.810922 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.811472 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.812027 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.812842 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.813044 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.813129 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.814260 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.814261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.814262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.814269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.815467 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.815850 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.815851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.815851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.815937 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.816188 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.816189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.816190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.816345 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.816453 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.816584 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.816660 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 12:21:32.819015 1784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:32.820648 1784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:32.827914 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.1.47.131:36827)
0925 12:21:32.823656 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.823660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.823669 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:32.823700 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.823702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.823715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.823777 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:32.824027 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825308 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.826707 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.827478 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.827563 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.827628 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.827770 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.827815 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260925 12:21:32.827912 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:36827 (local address 127.0.0.1:49586)
0925 12:21:32.823284 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.823327 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.823400 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.823813 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.823817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.823820 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:32.823928 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.823931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825413 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.825414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.826633 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.826636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.827575 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.827576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.827643 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.827654 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.827751 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.827806 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":7,"thread_start_us":59,"threads_started":1}
I20260925 12:21:32.828409 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:45963 (local address 127.0.0.1:42664)
0925 12:21:32.823443 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.823453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.823522 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.823952 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.823952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.823955 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:32.824007 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.824009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825509 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.825510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.826527 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.826530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.828185 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.828186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.828244 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.828248 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.828309 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.828346 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260925 12:21:32.828620 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.1.47.129:45963)
0925 12:21:32.823658 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.823662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.823672 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:32.823725 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.823726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.823739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.823953 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:32.824251 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825419 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.826621 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.827478 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.827564 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.827629 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.828506 (+ 877us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.828549 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":88,"threads_started":1}
I20260925 12:21:32.828742 2177 tablet_service.cc:1467] Processing CreateTablet for tablet c44130b8e6e54dab935b2be62813fffc (DEFAULT_TABLE table=test-table [id=44c7b1cbd991447eae448838bf6c6686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:32.828960 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41379 (local address 127.0.0.1:42480)
0925 12:21:32.824053 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.824064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.824116 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.824360 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.824360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.824362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:32.824395 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.824396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825262 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.825263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.826639 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.826640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.828671 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.828672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.828772 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.828777 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.828827 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.828872 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":27,"thread_start_us":56,"threads_started":1}
I20260925 12:21:32.829126 2177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44130b8e6e54dab935b2be62813fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:32.829311 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.823450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.1.47.130:41379)
0925 12:21:32.823610 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.823616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.824120 (+ 504us) server_negotiation.cc:407] Connection header received
0925 12:21:32.824180 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.824183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.824218 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.824293 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:32.824479 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.825163 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.827672 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.828499 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.828567 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.828633 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.829172 (+ 539us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.829235 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260925 12:21:32.829347 1909 tablet_service.cc:1467] Processing CreateTablet for tablet c44130b8e6e54dab935b2be62813fffc (DEFAULT_TABLE table=test-table [id=44c7b1cbd991447eae448838bf6c6686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:32.829650 1909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44130b8e6e54dab935b2be62813fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:32.829955 2043 tablet_service.cc:1467] Processing CreateTablet for tablet c44130b8e6e54dab935b2be62813fffc (DEFAULT_TABLE table=test-table [id=44c7b1cbd991447eae448838bf6c6686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:32.830276 2043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44130b8e6e54dab935b2be62813fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:32.832748 2260 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: Bootstrap starting.
I20260925 12:21:32.833181 2261 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: Bootstrap starting.
I20260925 12:21:32.833611 2260 tablet_bootstrap.cc:654] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:32.833781 2262 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Bootstrap starting.
I20260925 12:21:32.833807 2261 tablet_bootstrap.cc:654] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:32.834194 2260 log.cc:824] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:32.834422 2261 log.cc:824] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:32.834530 2262 tablet_bootstrap.cc:654] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:32.835028 2262 log.cc:824] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:32.835043 2260 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: No bootstrap required, opened a new log
I20260925 12:21:32.835132 2260 ts_tablet_manager.cc:1397] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:21:32.835759 2262 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: No bootstrap required, opened a new log
I20260925 12:21:32.835824 2262 ts_tablet_manager.cc:1397] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:32.837046 2261 tablet_bootstrap.cc:492] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: No bootstrap required, opened a new log
I20260925 12:21:32.837173 2261 ts_tablet_manager.cc:1397] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 12:21:32.837745 2260 raft_consensus.cc:348] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.837946 2260 raft_consensus.cc:374] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:32.837986 2260 raft_consensus.cc:729] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe56be4fd4834ef5a955fbcecd61a652, State: Initialized, Role: FOLLOWER
I20260925 12:21:32.838119 2260 consensus_queue.cc:260] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.838430 2243 heartbeater.cc:503] Master 127.1.47.190:35325 was elected leader, sending a full tablet report...
I20260925 12:21:32.838407 2262 raft_consensus.cc:348] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.838626 2262 raft_consensus.cc:374] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:32.838423 2260 ts_tablet_manager.cc:1428] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 12:21:32.838675 2262 raft_consensus.cc:729] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f870b120301a45eba1ac7480cb3880a3, State: Initialized, Role: FOLLOWER
I20260925 12:21:32.839250 2262 consensus_queue.cc:260] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.839359 2261 raft_consensus.cc:348] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.839564 2109 heartbeater.cc:503] Master 127.1.47.190:35325 was elected leader, sending a full tablet report...
I20260925 12:21:32.839557 2261 raft_consensus.cc:374] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:32.839609 2261 raft_consensus.cc:729] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff367707540c448aa7f2d99cdaab549e, State: Initialized, Role: FOLLOWER
I20260925 12:21:32.839740 2261 consensus_queue.cc:260] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.839815 2262 ts_tablet_manager.cc:1428] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 12:21:32.840080 2261 ts_tablet_manager.cc:1428] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:32.840111 1975 heartbeater.cc:503] Master 127.1.47.190:35325 was elected leader, sending a full tablet report...
I20260925 12:21:32.887602 2267 raft_consensus.cc:482] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 12:21:32.887795 2110 tablet.cc:2369] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:32.887769 2267 raft_consensus.cc:504] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.888181 2267 leader_election.cc:288] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe56be4fd4834ef5a955fbcecd61a652 (127.1.47.131:36827), ff367707540c448aa7f2d99cdaab549e (127.1.47.129:45963)
I20260925 12:21:32.891108 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.888444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:45963 (local address 127.1.47.130:49513)
0925 12:21:32.888515 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.888530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.888595 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.888760 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.888761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.888762 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:32.888837 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.888839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.889513 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.889514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.890285 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.890287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.890841 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.890924 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.890927 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.890991 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.891033 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 12:21:32.891108 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.888574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:49513 (local address 127.1.47.129:45963)
0925 12:21:32.888619 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.888621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.888630 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:32.888657 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.888658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.888670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.888743 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:32.888912 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.889415 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890384 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.890750 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890834 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.890843 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.891003 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.891038 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:21:32.891214 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.888558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:39481 (local address 127.1.47.131:36827)
0925 12:21:32.888616 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.888618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.888752 (+ 134us) server_negotiation.cc:407] Connection header received
0925 12:21:32.888830 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.888832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.888849 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.888914 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:32.889068 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.889515 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890464 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.890846 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890937 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.890945 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.891116 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.891154 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 12:21:32.891220 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.888444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:36827 (local address 127.1.47.130:39481)
0925 12:21:32.888658 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:32.888668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:32.888762 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:32.888917 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:32.888918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:32.888920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:32.888994 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.888996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.889613 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.889614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.890361 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:32.890363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.890937 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.890938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:32.891020 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.891023 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:32.891087 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:21:32.891129 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":24,"thread_start_us":119,"threads_started":1}
I20260925 12:21:32.891464 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c44130b8e6e54dab935b2be62813fffc" candidate_uuid: "f870b120301a45eba1ac7480cb3880a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff367707540c448aa7f2d99cdaab549e" is_pre_election: true
I20260925 12:21:32.891541 2197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c44130b8e6e54dab935b2be62813fffc" candidate_uuid: "f870b120301a45eba1ac7480cb3880a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe56be4fd4834ef5a955fbcecd61a652" is_pre_election: true
I20260925 12:21:32.891664 1929 raft_consensus.cc:2454] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f870b120301a45eba1ac7480cb3880a3 in term 0.
I20260925 12:21:32.891687 2197 raft_consensus.cc:2454] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f870b120301a45eba1ac7480cb3880a3 in term 0.
I20260925 12:21:32.891920 1996 leader_election.cc:302] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f870b120301a45eba1ac7480cb3880a3, ff367707540c448aa7f2d99cdaab549e; no voters:
I20260925 12:21:32.892105 2267 raft_consensus.cc:2781] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:32.892174 2267 raft_consensus.cc:482] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:32.892223 2267 raft_consensus.cc:3037] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:32.892917 2267 raft_consensus.cc:504] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.893059 2267 leader_election.cc:288] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [CANDIDATE]: Term 1 election: Requested vote from peers fe56be4fd4834ef5a955fbcecd61a652 (127.1.47.131:36827), ff367707540c448aa7f2d99cdaab549e (127.1.47.129:45963)
I20260925 12:21:32.893198 2197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c44130b8e6e54dab935b2be62813fffc" candidate_uuid: "f870b120301a45eba1ac7480cb3880a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe56be4fd4834ef5a955fbcecd61a652"
I20260925 12:21:32.893213 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c44130b8e6e54dab935b2be62813fffc" candidate_uuid: "f870b120301a45eba1ac7480cb3880a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff367707540c448aa7f2d99cdaab549e"
I20260925 12:21:32.893280 1929 raft_consensus.cc:3037] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:32.893303 2197 raft_consensus.cc:3037] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:32.894004 1929 raft_consensus.cc:2454] T c44130b8e6e54dab935b2be62813fffc P ff367707540c448aa7f2d99cdaab549e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f870b120301a45eba1ac7480cb3880a3 in term 1.
I20260925 12:21:32.894104 2197 raft_consensus.cc:2454] T c44130b8e6e54dab935b2be62813fffc P fe56be4fd4834ef5a955fbcecd61a652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f870b120301a45eba1ac7480cb3880a3 in term 1.
I20260925 12:21:32.894168 1996 leader_election.cc:302] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f870b120301a45eba1ac7480cb3880a3, ff367707540c448aa7f2d99cdaab549e; no voters:
I20260925 12:21:32.894274 2267 raft_consensus.cc:2781] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:32.894449 2267 raft_consensus.cc:686] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [term 1 LEADER]: Becoming Leader. State: Replica: f870b120301a45eba1ac7480cb3880a3, State: Running, Role: LEADER
I20260925 12:21:32.894557 2267 consensus_queue.cc:237] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } }
I20260925 12:21:32.895442 1783 catalog_manager.cc:5495] T c44130b8e6e54dab935b2be62813fffc P f870b120301a45eba1ac7480cb3880a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f870b120301a45eba1ac7480cb3880a3 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "f870b120301a45eba1ac7480cb3880a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe56be4fd4834ef5a955fbcecd61a652" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff367707540c448aa7f2d99cdaab549e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 45963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f870b120301a45eba1ac7480cb3880a3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41379 } health_report { overall_health: HEALTHY } } }
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:32.907193 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.899605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.1.47.129:45963)
0925 12:21:32.899646 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.899648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.899815 (+ 167us) server_negotiation.cc:407] Connection header received
0925 12:21:32.899940 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.899942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.899952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.900003 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.900568 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.901048 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.901780 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.901994 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.902089 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.904643 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.904653 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.904654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.904661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.906181 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.906511 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.906514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.906515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.906576 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.906747 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.906748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.906748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.906891 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.907035 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.907065 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.907125 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:32.931537 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.924148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.1.47.129:45963)
0925 12:21:32.924223 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.924226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.924333 (+ 107us) server_negotiation.cc:407] Connection header received
0925 12:21:32.924431 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.924432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.924443 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.924495 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.925071 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.925559 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.926317 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.926529 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.926619 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.928940 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.928942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.928942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.928950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.930387 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.930721 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.930722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.930723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.930774 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.930976 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.930977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.930977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.931096 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.931169 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.931353 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.931432 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 12:21:32 dist-test-slave-l9ww krb5kdc[1739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:32.954437 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:32.947209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.1.47.129:45963)
0925 12:21:32.947286 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:32.947288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:32.947422 (+ 134us) server_negotiation.cc:407] Connection header received
0925 12:21:32.947473 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:32.947474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:32.947483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:32.947538 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:32.948277 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.948743 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.949491 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:32.949669 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:32.949720 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:32.952085 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:32.952086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:32.952086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:32.952092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:32.953467 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.953812 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.953813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.953814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.953871 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:32.954040 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:32.954040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:32.954041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:32.954129 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:32.954209 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:32.954290 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 12:21:32.954342 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 12:21:32.955067 1889 tablet_service.cc:2908] Rejecting scan request for tablet c44130b8e6e54dab935b2be62813fffc: Uninitialized: clean time has not yet been initialized
I20260925 12:21:32.956267 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1846
I20260925 12:21:32.960881 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1980
I20260925 12:21:32.965503 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2114
I20260925 12:21:32.970474 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 1752
2026-09-25T12:21:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2055 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:33 dist-test-slave-l9ww krb5kdc[2284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:33 dist-test-slave-l9ww krb5kdc[2284](info): set up 2 sockets
Sep 25 12:21:33 dist-test-slave-l9ww krb5kdc[2284](info): commencing operation
krb5kdc: starting...
W20260925 12:21:34.019462 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:34.142530 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37639
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:34.225433 2297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:34.225745 2297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:34.225786 2297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:34.227029 2297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:34.227111 2297 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 12:21:34.227133 2297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:34.227155 2297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:34.227185 2297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:34.227214 2297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:34.229048 2297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37639
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:34.229378 2297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:34.229640 2297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:34.234874 2302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:34.234977 2297 server_base.cc:1034] running on GCE node
W20260925 12:21:34.234910 2305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.235128 2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:34.235448 2297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:34.235798 2297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:34.237033 2297 hybrid_clock.cc:648] HybridClock initialized: now 1790338894237000 us; error 55 us; skew 500 ppm
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:34.239954 2297 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:34.240477 2297 webserver.cc:460] Webserver started at http://127.1.47.190:46093/ using document root <none> and password file <none>
I20260925 12:21:34.240701 2297 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:34.240747 2297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:34.240847 2297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:34.241994 2297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "863440e22cf7468a854baaa9fdef6652"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "a2dd7de5c8350eab7f62b13b873096fd"
server_key_iv: "5be303db18b2d3433cd0d8b943e13da6"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.242321 2297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "863440e22cf7468a854baaa9fdef6652"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "a2dd7de5c8350eab7f62b13b873096fd"
server_key_iv: "5be303db18b2d3433cd0d8b943e13da6"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.243760 2297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 12:21:34.244796 2312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.245038 2297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:34.245155 2297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "863440e22cf7468a854baaa9fdef6652"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "a2dd7de5c8350eab7f62b13b873096fd"
server_key_iv: "5be303db18b2d3433cd0d8b943e13da6"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.245246 2297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:34.265357 2297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:34.268165 2297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:34.268296 2297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:34.273358 2297 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37639
I20260925 12:21:34.273460 2374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37639 every 8 connection(s)
I20260925 12:21:34.273854 2297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:34.274636 2375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:34.277191 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Bootstrap starting.
I20260925 12:21:34.277772 2375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:34.278013 2375 log.cc:824] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:34.278350 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2297
I20260925 12:21:34.278452 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:34.278630 1214 external_mini_cluster.cc:1428] Setting key 88f757cfe21f248155489b11ad1abcd7
I20260925 12:21:34.278796 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: No bootstrap required, opened a new log
I20260925 12:21:34.281745 2375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } }
I20260925 12:21:34.281893 2375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:34.281916 2375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863440e22cf7468a854baaa9fdef6652, State: Initialized, Role: FOLLOWER
I20260925 12:21:34.282056 2375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } }
I20260925 12:21:34.282140 2375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:34.282186 2375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:34.282239 2375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:34.282994 2375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } }
I20260925 12:21:34.283108 2375 leader_election.cc:302] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863440e22cf7468a854baaa9fdef6652; no voters:
I20260925 12:21:34.283273 2375 leader_election.cc:288] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:34.283339 2380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:34.283500 2380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [term 1 LEADER]: Becoming Leader. State: Replica: 863440e22cf7468a854baaa9fdef6652, State: Running, Role: LEADER
I20260925 12:21:34.283515 2375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:34.283610 2380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } }
I20260925 12:21:34.283995 2380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863440e22cf7468a854baaa9fdef6652. Latest consensus state: current_term: 1 leader_uuid: "863440e22cf7468a854baaa9fdef6652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } } }
I20260925 12:21:34.284098 2380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:34.284008 2382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863440e22cf7468a854baaa9fdef6652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863440e22cf7468a854baaa9fdef6652" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37639 } } }
I20260925 12:21:34.284269 2382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:34.284397 2389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:34.284992 2389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:34.286880 2397 mvcc.cc:205] Tried to move back new op lower bound from 7333228110995795968 to 7333228110986039296. Current Snapshot: MvccSnapshot[applied={T|T < 7333228110995795968}]
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:34.287396 2389 catalog_manager.cc:1347] Generated new cluster ID: f8782dd098d74ae3b26f4f86dbeaa44c
I20260925 12:21:34.287510 2389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:34.291693 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.279264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.1.47.190:37639)
0925 12:21:34.279459 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.279468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.279496 (+ 28us) server_negotiation.cc:407] Connection header received
0925 12:21:34.280242 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.280250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.280305 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.280438 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.281284 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.282505 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.283627 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.284384 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.284547 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.287737 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.287741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.287741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.287753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.289872 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.290313 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.290316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.290317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.290456 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.290686 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.290687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.290688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.291029 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.291206 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.291318 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.291467 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260925 12:21:34.293526 2389 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:34.294138 2389 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:34.297611 2389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Generated new TSK 0
I20260925 12:21:34.297814 2389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:34.326603 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37639
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:34.416199 2401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:34.416458 2401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:34.416503 2401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:34.418354 2401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:34.418437 2401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:34.418481 2401 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:34.418526 2401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:34.421523 2401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:34.421854 2401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:34.422153 2401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:34.429307 2406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.429301 2409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.429297 2407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:34.429581 2401 server_base.cc:1034] running on GCE node
I20260925 12:21:34.429801 2401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:34.430023 2401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:34.431169 2401 hybrid_clock.cc:648] HybridClock initialized: now 1790338894431156 us; error 31 us; skew 500 ppm
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:34.434041 2401 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:34.434602 2401 webserver.cc:460] Webserver started at http://127.1.47.129:39083/ using document root <none> and password file <none>
I20260925 12:21:34.434854 2401 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:34.434899 2401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:34.435038 2401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:34.436230 2401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "554b5acaf45c4e12a00f273b5c80dba7"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "55bea832f5f16504eddca57b2cbddbe8"
server_key_iv: "016e745e5cca2f4dadf4e859a0e4471c"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.436612 2401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "554b5acaf45c4e12a00f273b5c80dba7"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "55bea832f5f16504eddca57b2cbddbe8"
server_key_iv: "016e745e5cca2f4dadf4e859a0e4471c"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.437990 2401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:34.438832 2416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.439162 2401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:34.439275 2401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "554b5acaf45c4e12a00f273b5c80dba7"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "55bea832f5f16504eddca57b2cbddbe8"
server_key_iv: "016e745e5cca2f4dadf4e859a0e4471c"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.439353 2401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:34.455183 2401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:34.457487 2401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:34.457681 2401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:34.458228 2401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:34.458295 2401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.458346 2401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:34.458374 2401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.465513 2401 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:41313
I20260925 12:21:34.465585 2531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:41313 every 8 connection(s)
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:34.465898 2401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:34.468915 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.458784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:34599 (local address 127.1.47.190:37639)
0925 12:21:34.458858 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.458861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.459084 (+ 223us) server_negotiation.cc:407] Connection header received
0925 12:21:34.459570 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.459572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.459586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.459669 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.460653 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.461513 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.463133 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.463501 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.463589 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.466388 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.466389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.466389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.466396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.467557 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.468092 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.468093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.468094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.468149 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.468357 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.468357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.468358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.468459 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.468507 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.468772 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.468848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 12:21:34.469152 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.458752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.129:34599)
0925 12:21:34.458936 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.458977 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.459467 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.459783 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.459788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.460240 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.460537 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.460545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.461623 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.461624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.463047 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.463049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.463622 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.463623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.463744 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.464420 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.464432 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.466294 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.467653 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.467659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.467661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.468006 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.468196 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.468196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.468197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.468316 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.468516 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.468520 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.468646 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.468732 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.468875 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260925 12:21:34.469645 2532 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37639
I20260925 12:21:34.469734 2532 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:34.469929 2532 heartbeater.cc:511] Master 127.1.47.190:37639 requested a full tablet report, sending...
I20260925 12:21:34.470487 2328 ts_manager.cc:194] Registered new tserver with Master: 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313)
I20260925 12:21:34.471294 2328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:34599
I20260925 12:21:34.472288 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2401
I20260925 12:21:34.472369 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:34.472576 1214 external_mini_cluster.cc:1428] Setting key 7f948218dfdb4f2ec7f68f510697f1c2
I20260925 12:21:34.477070 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.470914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:44213 (local address 127.1.47.190:37639)
0925 12:21:34.470950 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.470953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.471083 (+ 130us) server_negotiation.cc:407] Connection header received
0925 12:21:34.471156 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.471156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.471166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.471234 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.471642 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.472438 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.473262 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.473469 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.473555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.474531 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.474533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.474533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.474541 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.475830 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.476235 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.476237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.476237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.476297 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.476496 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.476497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.476498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.476639 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.476709 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.476891 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.476981 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:21:34.477084 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.470852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.129:44213)
0925 12:21:34.471005 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.471016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.471099 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.471240 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.471241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.471478 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.471577 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.471579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.472538 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.472539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.473164 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.473166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.473560 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.473562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.473637 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.474151 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.474155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.474437 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.475960 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.475961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.475962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.476145 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.476365 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.476365 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.476366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.476425 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.476736 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.476737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.476788 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.476863 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.476975 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:34.498800 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37639
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:34.581286 2541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:34.581516 2541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:34.581543 2541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:34.582758 2541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:34.582808 2541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:34.582829 2541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:34.582849 2541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:34.584563 2541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:34.584875 2541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:34.585201 2541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:34.590396 2547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.590407 2546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.590440 2549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:34.590664 2541 server_base.cc:1034] running on GCE node
I20260925 12:21:34.590957 2541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:34.591241 2541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:34.592428 2541 hybrid_clock.cc:648] HybridClock initialized: now 1790338894592403 us; error 42 us; skew 500 ppm
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:34.595057 2541 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:34.595644 2541 webserver.cc:460] Webserver started at http://127.1.47.130:39865/ using document root <none> and password file <none>
I20260925 12:21:34.595914 2541 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:34.595974 2541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:34.596071 2541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:34.597116 2541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "c20d2da3a7a24a3d991b3f929624328d"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "93b73b2ee0c0c5e39d133ec2da587417"
server_key_iv: "74767ae4e77c2634b2a2218ea3abc631"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.597443 2541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "c20d2da3a7a24a3d991b3f929624328d"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "93b73b2ee0c0c5e39d133ec2da587417"
server_key_iv: "74767ae4e77c2634b2a2218ea3abc631"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.598760 2541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:34.599489 2556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.599649 2541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:34.599745 2541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "c20d2da3a7a24a3d991b3f929624328d"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "93b73b2ee0c0c5e39d133ec2da587417"
server_key_iv: "74767ae4e77c2634b2a2218ea3abc631"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.599803 2541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:34.616010 2541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:34.618481 2541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:34.618613 2541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:34.619081 2541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:34.619123 2541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.619148 2541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:34.619160 2541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.626355 2541 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:40577
I20260925 12:21:34.626434 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:40577 every 8 connection(s)
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:34.626756 2541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:34.630120 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.619687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:51965 (local address 127.1.47.190:37639)
0925 12:21:34.619758 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.619761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.620004 (+ 243us) server_negotiation.cc:407] Connection header received
0925 12:21:34.620500 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.620501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.620514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.620558 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.621442 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.622295 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.624131 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.624428 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.624524 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.627231 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.627233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.627234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.627243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.628668 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.629095 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.629097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.629097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.629161 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.629438 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.629439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.629440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.629566 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.629646 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.629943 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.630030 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 12:21:34.630333 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.619639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.130:51965)
0925 12:21:34.619806 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.619847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.620382 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.620633 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.620637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.621046 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.621349 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.621355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.622404 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.622406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.623992 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.623995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.624531 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.624532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.624678 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.625276 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.625288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.627138 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.628759 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.628765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.628768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.629011 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.629229 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.629230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.629231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.629347 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.629636 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.629643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.629792 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.629903 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.630070 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260925 12:21:34.630860 2672 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37639
I20260925 12:21:34.630981 2672 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:34.631201 2672 heartbeater.cc:511] Master 127.1.47.190:37639 requested a full tablet report, sending...
I20260925 12:21:34.631616 2328 ts_manager.cc:194] Registered new tserver with Master: c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577)
I20260925 12:21:34.632184 2328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:51965
I20260925 12:21:34.633880 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2541
I20260925 12:21:34.633985 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:34.634171 1214 external_mini_cluster.cc:1428] Setting key b99d1104caeaefc9b73914e8f0725e3d
I20260925 12:21:34.639214 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.632043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:60937 (local address 127.1.47.190:37639)
0925 12:21:34.632080 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.632082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.632172 (+ 90us) server_negotiation.cc:407] Connection header received
0925 12:21:34.632220 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.632221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.632231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.632303 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.632625 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.633255 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.633993 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.634258 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.634326 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.636390 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.636391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.636392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.636399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.638028 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.638373 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.638374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.638374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.638435 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.638644 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.638645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.638645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.638794 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.638876 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.639037 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.639122 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:34.639219 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.631985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.130:60937)
0925 12:21:34.632105 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.632116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.632173 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.632308 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.632308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.632505 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.632555 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.632557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.633343 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.633344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.633859 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.633860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.634327 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.634328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.634400 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.635937 (+ 1537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.635943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.636288 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.638111 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.638111 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.638112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.638286 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.638506 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.638506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.638507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.638570 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.638866 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.638867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.638917 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.639008 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.639119 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:34.659232 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37639
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:34.744444 2681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:34.744742 2681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:34.744786 2681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:34.745965 2681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:34.746032 2681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:34.746068 2681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:34.746104 2681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:34.747499 2681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:34.747761 2681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:34.748117 2681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:34.753237 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.753250 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:34.753250 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:34.753499 2681 server_base.cc:1034] running on GCE node
I20260925 12:21:34.753697 2681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:34.753971 2681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:34.755115 2681 hybrid_clock.cc:648] HybridClock initialized: now 1790338894755105 us; error 27 us; skew 500 ppm
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:34.757786 2681 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:34.758342 2681 webserver.cc:460] Webserver started at http://127.1.47.131:33213/ using document root <none> and password file <none>
I20260925 12:21:34.758579 2681 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:34.758648 2681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:34.758805 2681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:34.759989 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "e0caeda5af964f0a8858cf03f77adaed"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "097a908aa3e2dffc145f33cad39f48bc"
server_key_iv: "9723f1eedd1acf0f4cf2cdf269d2c2af"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.760365 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "e0caeda5af964f0a8858cf03f77adaed"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "097a908aa3e2dffc145f33cad39f48bc"
server_key_iv: "9723f1eedd1acf0f4cf2cdf269d2c2af"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.761782 2681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:21:34.762584 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.762754 2681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:34.762857 2681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "e0caeda5af964f0a8858cf03f77adaed"
format_stamp: "Formatted at 2026-09-25 12:21:34 on dist-test-slave-l9ww"
server_key: "097a908aa3e2dffc145f33cad39f48bc"
server_key_iv: "9723f1eedd1acf0f4cf2cdf269d2c2af"
server_key_version: "encryptionkey@0"
I20260925 12:21:34.762965 2681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:34.773280 2681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:34.775769 2681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:34.776051 2681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:34.776680 2681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:34.776746 2681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:34.776793 2681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:34.776839 2681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:34.784858 2681 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:44553
I20260925 12:21:34.784986 2811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:44553 every 8 connection(s)
I20260925 12:21:34.785336 2681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:34.789284 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.777125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:59243 (local address 127.1.47.190:37639)
0925 12:21:34.777196 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.777199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.777438 (+ 239us) server_negotiation.cc:407] Connection header received
0925 12:21:34.777971 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.777973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.777985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.778032 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.778886 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.779895 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.781025 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.781233 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.781325 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.786038 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.786040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.786040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.786049 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.787755 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.788262 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.788263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.788264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.788332 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.788603 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.788605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.788605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.788734 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.788813 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.789098 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.789188 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:21:34.789496 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.777134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.131:59243)
0925 12:21:34.777297 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.777338 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.777862 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.778116 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.778120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.778466 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.778767 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.778775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.780012 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.780013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.780917 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.780919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.781347 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.781348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.781469 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.782026 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.782034 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.784000 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.787879 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.787887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.787890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.788161 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.788402 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.788403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.788403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.788530 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.788803 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.788809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.788962 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.789077 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.789250 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 12:21:34.790055 2812 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37639
I20260925 12:21:34.790182 2812 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:34.790400 2812 heartbeater.cc:511] Master 127.1.47.190:37639 requested a full tablet report, sending...
I20260925 12:21:34.790812 2329 ts_manager.cc:194] Registered new tserver with Master: e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553)
I20260925 12:21:34.791232 2329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:59243
I20260925 12:21:34.794561 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2681
I20260925 12:21:34.794665 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:34.794874 1214 external_mini_cluster.cc:1428] Setting key 2350baa089c8f5d63e7519e0f9b56296
I20260925 12:21:34.795907 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:21:34.797137 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.791123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.1.47.131:44687)
0925 12:21:34.791232 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.791243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.791298 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.791446 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.791447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.791679 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:34.791739 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.791742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.792475 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.792476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.792985 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.792986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.793348 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.793375 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.793463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.793961 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:34.793966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:34.794265 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:34.796236 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.796237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.796238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.796401 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.796604 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:34.796604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:34.796604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:34.796647 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:34.796880 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:34.796880 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:34.796915 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.796960 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.797067 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":61,"threads_started":1}
I20260925 12:21:34.797207 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.791237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:44687 (local address 127.1.47.190:37639)
0925 12:21:34.791278 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.791280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.791288 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:34.791379 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.791380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.791389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.791433 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.791847 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.792385 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.793066 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.793257 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.793311 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.794419 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.794420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.794421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.794453 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.796168 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.796473 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.796474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.796475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.796540 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.796684 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.796685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.796685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.796824 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.796892 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.796989 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.797069 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 12:21:34 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:34.831250 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.822958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.1.47.190:37639)
0925 12:21:34.823024 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.823027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.823037 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:34.823066 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.823067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.823077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.823175 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:34.824212 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.824880 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.825853 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.826077 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.826152 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.828622 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:34.828623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:34.828624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:34.828631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:34.830021 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.830454 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.830455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.830455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.830526 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:34.830725 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:34.830726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:34.830726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:34.830897 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:34.830976 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.831103 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.831180 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:34.833103 2329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:34.834697 2329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:34.841086 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:41313 (local address 127.0.0.1:45364)
0925 12:21:34.837389 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.837399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.837437 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.837647 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.837651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.837653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:34.837759 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.837761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.838731 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.838732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.839806 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.839808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.840752 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.840753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.840833 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.840846 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.840942 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.840996 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":82,"threads_started":1}
I20260925 12:21:34.841094 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.1.47.129:41313)
0925 12:21:34.837446 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.837452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.837462 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:34.837504 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.837507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.837538 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.837629 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:34.837852 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.838640 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.839919 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.840662 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.840742 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.840814 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.840960 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.840996 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260925 12:21:34.841778 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:44553 (local address 127.0.0.1:37750)
0925 12:21:34.837677 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.837688 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.837742 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.838001 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.838001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.838003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:34.838031 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.838033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.839331 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.839332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.840429 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.840432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.841511 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.841512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.841599 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.841603 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.841665 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.841695 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":65,"threads_started":1}
I20260925 12:21:34.841988 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.1.47.131:44553)
0925 12:21:34.837679 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.837684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.837800 (+ 116us) server_negotiation.cc:407] Connection header received
0925 12:21:34.837840 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.837842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.837856 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.837940 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:34.838103 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.839241 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.840512 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.841432 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.841506 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.841566 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.841670 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.841715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260925 12:21:34.842074 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40577 (local address 127.0.0.1:45040)
0925 12:21:34.837249 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:34.837301 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:34.837370 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:34.837957 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:34.837958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:34.837959 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:34.838013 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.838015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.839367 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.839368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.840288 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:34.840291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.841807 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.841808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:34.841887 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.841892 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:34.841926 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 12:21:34.841971 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1,"thread_start_us":46,"threads_started":1}
I20260925 12:21:34.842427 2466 tablet_service.cc:1467] Processing CreateTablet for tablet a358fad05b494214bef053d98cd2d9f0 (DEFAULT_TABLE table=test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:34.842581 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:34.837193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.1.47.130:40577)
0925 12:21:34.837788 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:34.837793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:34.837802 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:34.837839 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:34.837841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:34.837856 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:34.838087 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:34.838143 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.839231 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.840375 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:34.841127 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:34.842365 (+ 1238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:34.842445 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:34.842468 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 12:21:34.842511 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":96,"threads_started":1}
I20260925 12:21:34.842533 2746 tablet_service.cc:1467] Processing CreateTablet for tablet a358fad05b494214bef053d98cd2d9f0 (DEFAULT_TABLE table=test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:34.842840 2466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a358fad05b494214bef053d98cd2d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:34.842840 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a358fad05b494214bef053d98cd2d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:34.844477 2606 tablet_service.cc:1467] Processing CreateTablet for tablet a358fad05b494214bef053d98cd2d9f0 (DEFAULT_TABLE table=test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:34.844789 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a358fad05b494214bef053d98cd2d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:34.846539 2835 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Bootstrap starting.
I20260925 12:21:34.846614 2836 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Bootstrap starting.
I20260925 12:21:34.847234 2836 tablet_bootstrap.cc:654] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:34.847602 2836 log.cc:824] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:34.847710 2835 tablet_bootstrap.cc:654] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:34.848100 2835 log.cc:824] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:34.848340 2838 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Bootstrap starting.
I20260925 12:21:34.848481 2836 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: No bootstrap required, opened a new log
I20260925 12:21:34.848551 2836 ts_tablet_manager.cc:1397] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:34.849082 2838 tablet_bootstrap.cc:654] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:34.849468 2838 log.cc:824] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:34.850236 2838 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: No bootstrap required, opened a new log
I20260925 12:21:34.850319 2838 ts_tablet_manager.cc:1397] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:34.850728 2836 raft_consensus.cc:348] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.850922 2836 raft_consensus.cc:374] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:34.850956 2836 raft_consensus.cc:729] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554b5acaf45c4e12a00f273b5c80dba7, State: Initialized, Role: FOLLOWER
I20260925 12:21:34.851071 2836 consensus_queue.cc:260] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.851380 2836 ts_tablet_manager.cc:1428] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:34.851506 2532 heartbeater.cc:503] Master 127.1.47.190:37639 was elected leader, sending a full tablet report...
I20260925 12:21:34.852357 2835 tablet_bootstrap.cc:492] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: No bootstrap required, opened a new log
I20260925 12:21:34.852434 2835 ts_tablet_manager.cc:1397] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 12:21:34.852483 2838 raft_consensus.cc:348] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.852640 2838 raft_consensus.cc:374] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:34.852671 2838 raft_consensus.cc:729] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20d2da3a7a24a3d991b3f929624328d, State: Initialized, Role: FOLLOWER
I20260925 12:21:34.852803 2838 consensus_queue.cc:260] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.853076 2838 ts_tablet_manager.cc:1428] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:34.853127 2672 heartbeater.cc:503] Master 127.1.47.190:37639 was elected leader, sending a full tablet report...
I20260925 12:21:34.854447 2835 raft_consensus.cc:348] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.854574 2835 raft_consensus.cc:374] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:34.854595 2835 raft_consensus.cc:729] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0caeda5af964f0a8858cf03f77adaed, State: Initialized, Role: FOLLOWER
I20260925 12:21:34.854710 2835 consensus_queue.cc:260] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:34.855050 2835 ts_tablet_manager.cc:1428] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:21:34.855170 2812 heartbeater.cc:503] Master 127.1.47.190:37639 was elected leader, sending a full tablet report...
W20260925 12:21:34.877384 2673 tablet.cc:2369] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:21:34.966682 2533 tablet.cc:2369] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:21:35.036074 2813 tablet.cc:2369] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:35.161803 2842 raft_consensus.cc:482] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:35.161949 2842 raft_consensus.cc:504] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.162461 2842 leader_election.cc:288] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313), e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553)
I20260925 12:21:35.165880 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.162798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:56701 (local address 127.1.47.129:41313)
0925 12:21:35.162850 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.162853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.163130 (+ 277us) server_negotiation.cc:407] Connection header received
0925 12:21:35.163160 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.163161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.163176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.163234 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.163419 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.164061 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.165016 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.165488 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.165571 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.165580 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.165758 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.165801 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:21:35.165885 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.162783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:49627 (local address 127.1.47.131:44553)
0925 12:21:35.162850 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.162853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.162975 (+ 122us) server_negotiation.cc:407] Connection header received
0925 12:21:35.163015 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.163017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.163032 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.163094 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.163271 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.163805 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.164949 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.165433 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.165542 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.165551 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.165721 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.165763 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:21:35.165971 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.162688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:44553 (local address 127.1.47.130:49627)
0925 12:21:35.162842 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.162855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.162933 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.163122 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.163123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.163126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.163192 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.163194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.163920 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.163921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.164856 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.164858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.165561 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.165562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.165646 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.165649 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.165698 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.165771 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260925 12:21:35.165984 2845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.162715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:41313 (local address 127.1.47.130:56701)
0925 12:21:35.163004 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.163015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.163085 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.163270 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.163271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.163273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.163336 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.163338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.164158 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.164159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.164916 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.164919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.165605 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.165606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.165659 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.165662 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.165733 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.165773 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":55,"threads_started":1}
I20260925 12:21:35.166354 2766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a358fad05b494214bef053d98cd2d9f0" candidate_uuid: "c20d2da3a7a24a3d991b3f929624328d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0caeda5af964f0a8858cf03f77adaed" is_pre_election: true
I20260925 12:21:35.166553 2766 raft_consensus.cc:2454] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c20d2da3a7a24a3d991b3f929624328d in term 0.
I20260925 12:21:35.166538 2486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a358fad05b494214bef053d98cd2d9f0" candidate_uuid: "c20d2da3a7a24a3d991b3f929624328d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554b5acaf45c4e12a00f273b5c80dba7" is_pre_election: true
I20260925 12:21:35.166723 2486 raft_consensus.cc:2454] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c20d2da3a7a24a3d991b3f929624328d in term 0.
I20260925 12:21:35.166810 2558 leader_election.cc:302] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c20d2da3a7a24a3d991b3f929624328d, e0caeda5af964f0a8858cf03f77adaed; no voters:
I20260925 12:21:35.166937 2842 raft_consensus.cc:2781] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:35.167029 2842 raft_consensus.cc:482] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:35.167070 2842 raft_consensus.cc:3037] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.167848 2842 raft_consensus.cc:504] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.168095 2842 leader_election.cc:288] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [CANDIDATE]: Term 1 election: Requested vote from peers 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313), e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553)
I20260925 12:21:35.168255 2766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a358fad05b494214bef053d98cd2d9f0" candidate_uuid: "c20d2da3a7a24a3d991b3f929624328d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0caeda5af964f0a8858cf03f77adaed"
I20260925 12:21:35.168264 2486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a358fad05b494214bef053d98cd2d9f0" candidate_uuid: "c20d2da3a7a24a3d991b3f929624328d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554b5acaf45c4e12a00f273b5c80dba7"
I20260925 12:21:35.168347 2766 raft_consensus.cc:3037] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.168368 2486 raft_consensus.cc:3037] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.169108 2766 raft_consensus.cc:2454] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c20d2da3a7a24a3d991b3f929624328d in term 1.
I20260925 12:21:35.169229 2486 raft_consensus.cc:2454] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c20d2da3a7a24a3d991b3f929624328d in term 1.
I20260925 12:21:35.169282 2558 leader_election.cc:302] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c20d2da3a7a24a3d991b3f929624328d, e0caeda5af964f0a8858cf03f77adaed; no voters:
I20260925 12:21:35.169406 2842 raft_consensus.cc:2781] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:35.169529 2842 raft_consensus.cc:686] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 1 LEADER]: Becoming Leader. State: Replica: c20d2da3a7a24a3d991b3f929624328d, State: Running, Role: LEADER
I20260925 12:21:35.169646 2842 consensus_queue.cc:237] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.170516 2329 catalog_manager.cc:5495] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d reported cstate change: term changed from 0 to 1, leader changed from <none> to c20d2da3a7a24a3d991b3f929624328d (127.1.47.130). New cstate: current_term: 1 leader_uuid: "c20d2da3a7a24a3d991b3f929624328d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:35.196204 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.193505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.1.47.130:40577)
0925 12:21:35.193576 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.193579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.193590 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:35.193659 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.193659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.193680 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.193746 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:35.193922 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.194447 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.195365 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.195555 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.195619 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.195815 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.195992 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.196070 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.196119 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.196142 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:21:35.198175 2486 raft_consensus.cc:1261] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Refusing update from remote peer c20d2da3a7a24a3d991b3f929624328d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.198179 2766 raft_consensus.cc:1261] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Refusing update from remote peer c20d2da3a7a24a3d991b3f929624328d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.198555 2842 consensus_queue.cc:1035] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.198725 2846 consensus_queue.cc:1035] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [LEADER]: Connected to new peer: Peer: permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.201004 2849 mvcc.cc:205] Tried to move back new op lower bound from 7333228114728402944 to 7333228114615586816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228114728402944}]
I20260925 12:21:35.201156 2850 mvcc.cc:205] Tried to move back new op lower bound from 7333228114728402944 to 7333228114615586816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228114728402944}]
I20260925 12:21:35.201809 2851 mvcc.cc:205] Tried to move back new op lower bound from 7333228114728402944 to 7333228114615586816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228114728402944}]
I20260925 12:21:35.205363 2329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:35.205456 2329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:35.206359 2329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Sending DeleteTablet for 3 replicas of tablet a358fad05b494214bef053d98cd2d9f0
I20260925 12:21:35.206745 2606 tablet_service.cc:1514] Processing DeleteTablet for tablet a358fad05b494214bef053d98cd2d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:45040
I20260925 12:21:35.206786 2746 tablet_service.cc:1514] Processing DeleteTablet for tablet a358fad05b494214bef053d98cd2d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:37750
I20260925 12:21:35.207135 2860 tablet_replica.cc:330] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: stopping tablet replica
I20260925 12:21:35.207147 2466 tablet_service.cc:1514] Processing DeleteTablet for tablet a358fad05b494214bef053d98cd2d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:45364
I20260925 12:21:35.207259 2860 raft_consensus.cc:2229] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:35.207443 2860 raft_consensus.cc:2258] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.207499 2861 tablet_replica.cc:330] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: stopping tablet replica
I20260925 12:21:35.207599 2861 raft_consensus.cc:2229] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:35.207715 2861 raft_consensus.cc:2258] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.207645 2329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:35.207791 2329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:35.207995 2860 ts_tablet_manager.cc:1905] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.208200 2861 ts_tablet_manager.cc:1905] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.208753 2859 tablet_replica.cc:330] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: stopping tablet replica
I20260925 12:21:35.208868 2859 raft_consensus.cc:2229] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:35.208992 2859 raft_consensus.cc:2258] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.209539 2606 tablet_service.cc:1467] Processing CreateTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c (DEFAULT_TABLE table=test-table [id=48da8a9406de435993af6f3b6dc5986d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:35.209563 2466 tablet_service.cc:1467] Processing CreateTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c (DEFAULT_TABLE table=test-table [id=48da8a9406de435993af6f3b6dc5986d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:35.209582 2746 tablet_service.cc:1467] Processing CreateTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c (DEFAULT_TABLE table=test-table [id=48da8a9406de435993af6f3b6dc5986d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:35.209653 2466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f0e9b81cc84bbc85ce714097f7d11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.209654 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f0e9b81cc84bbc85ce714097f7d11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.209699 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f0e9b81cc84bbc85ce714097f7d11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.211020 2836 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Bootstrap starting.
I20260925 12:21:35.211038 2838 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Bootstrap starting.
I20260925 12:21:35.211700 2860 ts_tablet_manager.cc:1918] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:35.211772 2860 log.cc:1196] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/a358fad05b494214bef053d98cd2d9f0
I20260925 12:21:35.211843 2836 tablet_bootstrap.cc:654] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.211902 2838 tablet_bootstrap.cc:654] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.212100 2860 ts_tablet_manager.cc:1939] T a358fad05b494214bef053d98cd2d9f0 P c20d2da3a7a24a3d991b3f929624328d: Deleting consensus metadata
I20260925 12:21:35.212208 2835 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Bootstrap starting.
I20260925 12:21:35.212576 2314 catalog_manager.cc:4841] TS c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577): tablet a358fad05b494214bef053d98cd2d9f0 (table test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]) successfully deleted
I20260925 12:21:35.212627 2835 tablet_bootstrap.cc:654] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.212934 2861 ts_tablet_manager.cc:1918] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:35.213052 2861 log.cc:1196] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/a358fad05b494214bef053d98cd2d9f0
I20260925 12:21:35.213227 2835 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: No bootstrap required, opened a new log
I20260925 12:21:35.213266 2835 ts_tablet_manager.cc:1397] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:35.213342 2861 ts_tablet_manager.cc:1939] T a358fad05b494214bef053d98cd2d9f0 P 554b5acaf45c4e12a00f273b5c80dba7: Deleting consensus metadata
I20260925 12:21:35.213371 2835 raft_consensus.cc:348] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.213449 2835 raft_consensus.cc:374] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.213508 2835 raft_consensus.cc:729] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0caeda5af964f0a8858cf03f77adaed, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.213555 2835 consensus_queue.cc:260] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.213645 2835 ts_tablet_manager.cc:1428] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:35.213770 2313 catalog_manager.cc:4841] TS 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313): tablet a358fad05b494214bef053d98cd2d9f0 (table test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]) successfully deleted
I20260925 12:21:35.216342 2859 ts_tablet_manager.cc:1905] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.216471 2836 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: No bootstrap required, opened a new log
I20260925 12:21:35.216537 2836 ts_tablet_manager.cc:1397] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 12:21:35.216645 2838 tablet_bootstrap.cc:492] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: No bootstrap required, opened a new log
I20260925 12:21:35.216698 2838 ts_tablet_manager.cc:1397] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 12:21:35.216702 2836 raft_consensus.cc:348] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.216801 2836 raft_consensus.cc:374] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.216831 2836 raft_consensus.cc:729] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554b5acaf45c4e12a00f273b5c80dba7, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.216885 2836 consensus_queue.cc:260] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.216874 2838 raft_consensus.cc:348] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.216955 2838 raft_consensus.cc:374] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.216979 2838 raft_consensus.cc:729] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20d2da3a7a24a3d991b3f929624328d, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.217031 2838 consensus_queue.cc:260] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.217131 2838 ts_tablet_manager.cc:1428] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:35.217242 2836 ts_tablet_manager.cc:1428] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:35.218552 2859 ts_tablet_manager.cc:1918] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:35.218628 2859 log.cc:1196] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/a358fad05b494214bef053d98cd2d9f0
I20260925 12:21:35.218873 2859 ts_tablet_manager.cc:1939] T a358fad05b494214bef053d98cd2d9f0 P e0caeda5af964f0a8858cf03f77adaed: Deleting consensus metadata
I20260925 12:21:35.219450 2314 catalog_manager.cc:4841] TS e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553): tablet a358fad05b494214bef053d98cd2d9f0 (table test-table [id=7aea6526d98f4f90bdfa95d0b4b54814]) successfully deleted
I20260925 12:21:35.255133 2843 raft_consensus.cc:482] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:35.255260 2843 raft_consensus.cc:504] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.255673 2843 leader_election.cc:288] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313), c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577)
I20260925 12:21:35.260042 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.256132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:49007 (local address 127.1.47.129:41313)
0925 12:21:35.256172 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.256174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.256183 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:35.256213 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.256213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.256223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.256280 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.256496 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.257242 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.258927 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.259545 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.259628 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.259637 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.259911 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.259960 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:21:35.260064 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.255938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:41313 (local address 127.1.47.131:49007)
0925 12:21:35.256042 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.256056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.256113 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.256331 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.256332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.256334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.256405 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.256408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.257313 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.257314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.258798 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.258802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.259648 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.259649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.259746 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.259751 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.259840 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.259918 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":47}
I20260925 12:21:35.260068 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.256072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40577 (local address 127.1.47.131:58281)
0925 12:21:35.256201 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.256220 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.256293 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.256471 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.256472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.256475 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.256561 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.256563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.257264 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.257265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.258798 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.258802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.259680 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.259681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.259756 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.259759 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.259842 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.259924 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":90,"thread_start_us":62,"threads_started":1}
I20260925 12:21:35.260535 2486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f0e9b81cc84bbc85ce714097f7d11c" candidate_uuid: "e0caeda5af964f0a8858cf03f77adaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554b5acaf45c4e12a00f273b5c80dba7" is_pre_election: true
I20260925 12:21:35.260665 2486 raft_consensus.cc:2454] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0caeda5af964f0a8858cf03f77adaed in term 0.
I20260925 12:21:35.260857 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.256019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:58281 (local address 127.1.47.130:40577)
0925 12:21:35.256077 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.256079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.256285 (+ 206us) server_negotiation.cc:407] Connection header received
0925 12:21:35.256370 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.256371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.256391 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.256454 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.256641 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.257150 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.258927 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.259544 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.259602 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.259613 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.259841 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.260785 (+ 944us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 12:21:35.260874 2697 leader_election.cc:302] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 554b5acaf45c4e12a00f273b5c80dba7, e0caeda5af964f0a8858cf03f77adaed; no voters:
I20260925 12:21:35.261075 2843 raft_consensus.cc:2781] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:35.261125 2843 raft_consensus.cc:482] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:35.261148 2843 raft_consensus.cc:3037] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.261085 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f0e9b81cc84bbc85ce714097f7d11c" candidate_uuid: "e0caeda5af964f0a8858cf03f77adaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d2da3a7a24a3d991b3f929624328d" is_pre_election: true
I20260925 12:21:35.261214 2626 raft_consensus.cc:2454] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0caeda5af964f0a8858cf03f77adaed in term 0.
I20260925 12:21:35.261960 2843 raft_consensus.cc:504] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.262132 2843 leader_election.cc:288] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [CANDIDATE]: Term 1 election: Requested vote from peers 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313), c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577)
I20260925 12:21:35.262310 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f0e9b81cc84bbc85ce714097f7d11c" candidate_uuid: "e0caeda5af964f0a8858cf03f77adaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d2da3a7a24a3d991b3f929624328d"
I20260925 12:21:35.262420 2626 raft_consensus.cc:3037] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.263481 2626 raft_consensus.cc:2454] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0caeda5af964f0a8858cf03f77adaed in term 1.
I20260925 12:21:35.263628 2486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f0e9b81cc84bbc85ce714097f7d11c" candidate_uuid: "e0caeda5af964f0a8858cf03f77adaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554b5acaf45c4e12a00f273b5c80dba7"
I20260925 12:21:35.263736 2486 raft_consensus.cc:3037] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.263722 2698 leader_election.cc:302] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c20d2da3a7a24a3d991b3f929624328d, e0caeda5af964f0a8858cf03f77adaed; no voters:
I20260925 12:21:35.263832 2843 raft_consensus.cc:2781] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:35.263986 2843 raft_consensus.cc:686] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 1 LEADER]: Becoming Leader. State: Replica: e0caeda5af964f0a8858cf03f77adaed, State: Running, Role: LEADER
I20260925 12:21:35.264060 2843 consensus_queue.cc:237] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.264683 2486 raft_consensus.cc:2454] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0caeda5af964f0a8858cf03f77adaed in term 1.
I20260925 12:21:35.264997 2328 catalog_manager.cc:5495] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed reported cstate change: term changed from 0 to 1, leader changed from <none> to e0caeda5af964f0a8858cf03f77adaed (127.1.47.131). New cstate: current_term: 1 leader_uuid: "e0caeda5af964f0a8858cf03f77adaed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } health_report { overall_health: HEALTHY } } }
Sep 25 12:21:35 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:35.310076 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.296851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.1.47.190:37639)
0925 12:21:35.296930 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.296932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.297059 (+ 127us) server_negotiation.cc:407] Connection header received
0925 12:21:35.297134 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.297135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.297146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.297215 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:35.297706 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.298190 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.298910 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.299090 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.299174 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.306847 (+ 7673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:35.306849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:35.306849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:35.306857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:35.308559 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.309113 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.309114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.309115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.309176 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.309422 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.309423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.309423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.309539 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:35.309629 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.309862 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.309973 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":51}
I20260925 12:21:35.310083 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.296788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37639 (local address 127.0.0.1:51516)
0925 12:21:35.296969 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.296988 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.297077 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.297234 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.297235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.297576 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:35.297620 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.297622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.298275 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.298276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.298804 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.298805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.299175 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.299176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.299226 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.305009 (+ 5783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:35.305015 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:35.306713 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:35.308683 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:35.308695 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:35.308697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:35.308998 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:35.309267 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:35.309268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:35.309269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:35.309355 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:35.309627 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:35.309635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:35.309692 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.309773 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.309914 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260925 12:21:35.312613 2328 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:51516:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 12:21:35.312892 2328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:35.314414 2466 tablet_service.cc:1467] Processing CreateTablet for tablet be9c66bdea1c40cea5975f424030671c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4f1fc960e234b4f927e06914a2b5db0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:35.314448 2746 tablet_service.cc:1467] Processing CreateTablet for tablet be9c66bdea1c40cea5975f424030671c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4f1fc960e234b4f927e06914a2b5db0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:35.314584 2466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9c66bdea1c40cea5975f424030671c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.314594 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9c66bdea1c40cea5975f424030671c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.315689 2836 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: Bootstrap starting.
I20260925 12:21:35.315797 2606 tablet_service.cc:1467] Processing CreateTablet for tablet be9c66bdea1c40cea5975f424030671c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4f1fc960e234b4f927e06914a2b5db0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:35.316105 2835 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: Bootstrap starting.
I20260925 12:21:35.316129 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9c66bdea1c40cea5975f424030671c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:35.316138 2836 tablet_bootstrap.cc:654] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.316485 2835 tablet_bootstrap.cc:654] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.317170 2835 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: No bootstrap required, opened a new log
I20260925 12:21:35.317226 2835 ts_tablet_manager.cc:1397] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:35.317373 2835 raft_consensus.cc:348] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.317456 2835 raft_consensus.cc:374] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.317493 2835 raft_consensus.cc:729] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0caeda5af964f0a8858cf03f77adaed, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.317498 2838 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: Bootstrap starting.
I20260925 12:21:35.317536 2835 consensus_queue.cc:260] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.317623 2835 ts_tablet_manager.cc:1428] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:35.317754 2866 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.317818 2866 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.317880 2838 tablet_bootstrap.cc:654] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:35.318408 2836 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: No bootstrap required, opened a new log
I20260925 12:21:35.318450 2836 ts_tablet_manager.cc:1397] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 12:21:35.318573 2836 raft_consensus.cc:348] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.318621 2836 raft_consensus.cc:374] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.318634 2836 raft_consensus.cc:729] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554b5acaf45c4e12a00f273b5c80dba7, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.318694 2836 consensus_queue.cc:260] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.318789 2836 ts_tablet_manager.cc:1428] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:35.318841 2841 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.318945 2841 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.319202 2838 tablet_bootstrap.cc:492] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: No bootstrap required, opened a new log
I20260925 12:21:35.319245 2838 ts_tablet_manager.cc:1397] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:21:35.319819 2838 raft_consensus.cc:348] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.320065 2838 raft_consensus.cc:374] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:35.320101 2838 raft_consensus.cc:729] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20d2da3a7a24a3d991b3f929624328d, State: Initialized, Role: FOLLOWER
I20260925 12:21:35.320152 2838 consensus_queue.cc:260] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.320250 2838 ts_tablet_manager.cc:1428] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 12:21:35.320279 2854 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.320358 2854 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.566735 2841 raft_consensus.cc:482] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:35.566848 2841 raft_consensus.cc:504] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.567209 2841 leader_election.cc:288] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577), e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553)
I20260925 12:21:35.570479 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.567520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40577 (local address 127.1.47.129:40887)
0925 12:21:35.567682 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.567693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.567754 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.567960 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.567961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.567964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.568025 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.568027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.568675 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.568676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.569587 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.569589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.570198 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.570199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.570265 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.570269 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.570347 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.570389 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260925 12:21:35.570464 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.567609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:40887 (local address 127.1.47.130:40577)
0925 12:21:35.567773 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.567776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.567796 (+ 20us) server_negotiation.cc:407] Connection header received
0925 12:21:35.567846 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.567847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.567886 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.567938 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.568101 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.568582 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.569696 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.570063 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.570180 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.570191 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.570359 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.570397 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 12:21:35.570815 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9c66bdea1c40cea5975f424030671c" candidate_uuid: "554b5acaf45c4e12a00f273b5c80dba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d2da3a7a24a3d991b3f929624328d" is_pre_election: true
I20260925 12:21:35.570902 2626 raft_consensus.cc:2454] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 554b5acaf45c4e12a00f273b5c80dba7 in term 0.
I20260925 12:21:35.571094 2418 leader_election.cc:302] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 554b5acaf45c4e12a00f273b5c80dba7, c20d2da3a7a24a3d991b3f929624328d; no voters:
I20260925 12:21:35.571235 2841 raft_consensus.cc:2781] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:35.571285 2841 raft_consensus.cc:482] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:35.571290 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.567906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:47665 (local address 127.1.47.131:44553)
0925 12:21:35.567956 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.567958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.568076 (+ 118us) server_negotiation.cc:407] Connection header received
0925 12:21:35.568136 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.568138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.568153 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.568203 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:35.568348 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.568814 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.570316 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.570950 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.571031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.571039 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.571160 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.571214 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:21:35.571318 2841 raft_consensus.cc:3037] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.571380 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.567788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:44553 (local address 127.1.47.129:47665)
0925 12:21:35.568006 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.568014 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.568081 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.568205 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.568206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.568215 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:35.568276 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.568277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.568918 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.568919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.570223 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.570225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.571027 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.571028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.571090 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.571094 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.571155 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.571193 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":10,"thread_start_us":93,"threads_started":1}
I20260925 12:21:35.572153 2766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9c66bdea1c40cea5975f424030671c" candidate_uuid: "554b5acaf45c4e12a00f273b5c80dba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0caeda5af964f0a8858cf03f77adaed" is_pre_election: true
I20260925 12:21:35.572187 2841 raft_consensus.cc:504] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.572254 2766 raft_consensus.cc:2454] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 554b5acaf45c4e12a00f273b5c80dba7 in term 0.
I20260925 12:21:35.572307 2841 leader_election.cc:288] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [CANDIDATE]: Term 1 election: Requested vote from peers c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577), e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553)
I20260925 12:21:35.572540 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9c66bdea1c40cea5975f424030671c" candidate_uuid: "554b5acaf45c4e12a00f273b5c80dba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d2da3a7a24a3d991b3f929624328d"
I20260925 12:21:35.572619 2626 raft_consensus.cc:3037] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.573233 2626 raft_consensus.cc:2454] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 554b5acaf45c4e12a00f273b5c80dba7 in term 1.
I20260925 12:21:35.573421 2418 leader_election.cc:302] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 554b5acaf45c4e12a00f273b5c80dba7, c20d2da3a7a24a3d991b3f929624328d; no voters:
I20260925 12:21:35.573398 2766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9c66bdea1c40cea5975f424030671c" candidate_uuid: "554b5acaf45c4e12a00f273b5c80dba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0caeda5af964f0a8858cf03f77adaed"
I20260925 12:21:35.573472 2766 raft_consensus.cc:3037] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:35.573503 2841 raft_consensus.cc:2781] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:35.573634 2841 raft_consensus.cc:686] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 LEADER]: Becoming Leader. State: Replica: 554b5acaf45c4e12a00f273b5c80dba7, State: Running, Role: LEADER
I20260925 12:21:35.573701 2841 consensus_queue.cc:237] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } }
I20260925 12:21:35.574249 2766 raft_consensus.cc:2454] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 554b5acaf45c4e12a00f273b5c80dba7 in term 1.
I20260925 12:21:35.574231 2879 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: TxnStatusTablet state changed. Reason: New leader 554b5acaf45c4e12a00f273b5c80dba7. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.574316 2879 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:35.574471 2329 catalog_manager.cc:5495] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:35.574699 2881 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 12:21:35.574848 2881 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 12:21:35.608752 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.605882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.1.47.129:41313)
0925 12:21:35.605956 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.605958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.605969 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:35.606002 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.606004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.606025 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.606101 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:35.606222 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.606990 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.607759 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.608090 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.608161 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.608327 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.608458 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.608534 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.608647 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.608667 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:21:35.608743 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.605655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:41313 (local address 127.0.0.1:45370)
0925 12:21:35.605734 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.605746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.605792 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.606112 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.606113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.606114 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:35.606163 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.606164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.607076 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.607077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.607668 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.607670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.608162 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.608163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.608208 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.608226 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.608557 (+ 331us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.608573 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.608627 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.608676 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:21:35.609961 2766 raft_consensus.cc:1261] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Refusing update from remote peer 554b5acaf45c4e12a00f273b5c80dba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.610100 2626 raft_consensus.cc:1261] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Refusing update from remote peer 554b5acaf45c4e12a00f273b5c80dba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.610236 2879 consensus_queue.cc:1035] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.610364 2841 consensus_queue.cc:1035] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.610935 2866 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: TxnStatusTablet state changed. Reason: New leader 554b5acaf45c4e12a00f273b5c80dba7. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.610966 2854 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: TxnStatusTablet state changed. Reason: New leader 554b5acaf45c4e12a00f273b5c80dba7. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.611025 2866 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.611050 2854 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.611989 2879 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.612082 2879 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:35.612241 2854 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.612326 2854 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P c20d2da3a7a24a3d991b3f929624328d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.612303 2879 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.612375 2879 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P 554b5acaf45c4e12a00f273b5c80dba7: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:35.612928 2866 tablet_replica.cc:439] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "554b5acaf45c4e12a00f273b5c80dba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 } } peers { permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 } } peers { permanent_uuid: "e0caeda5af964f0a8858cf03f77adaed" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 44553 } } }
I20260925 12:21:35.613010 2866 tablet_replica.cc:442] T be9c66bdea1c40cea5975f424030671c P e0caeda5af964f0a8858cf03f77adaed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:35.616320 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.613975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.1.47.131:44553)
0925 12:21:35.614017 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.614020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.614028 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:35.614065 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.614066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.614084 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.614174 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:35.614287 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.614759 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.615477 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.615686 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.615772 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.615955 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.616067 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.616154 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.616195 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.616212 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:21:35.622246 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.619090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54555 (local address 127.1.47.129:41313)
0925 12:21:35.619146 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.619148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.619262 (+ 114us) server_negotiation.cc:407] Connection header received
0925 12:21:35.619290 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.619291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.619301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.619367 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:35.619479 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.620213 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.621470 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.621754 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.621826 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.622024 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.622055 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.622108 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.622155 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.622173 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:21:35.622262 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.619008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:41313 (local address 127.1.47.131:54555)
0925 12:21:35.619177 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.619196 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.619250 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.619369 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.619369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.619370 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:35.619417 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.619418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.620312 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.620313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.621368 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.621370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.621850 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.621851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.621911 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.621931 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.622116 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.622118 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.622153 (+ 35us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.622194 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260925 12:21:35.624706 2626 raft_consensus.cc:1261] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Refusing update from remote peer e0caeda5af964f0a8858cf03f77adaed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.624732 2486 raft_consensus.cc:1261] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Refusing update from remote peer e0caeda5af964f0a8858cf03f77adaed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:35.625017 2866 consensus_queue.cc:1035] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [LEADER]: Connected to new peer: Peer: permanent_uuid: "554b5acaf45c4e12a00f273b5c80dba7" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.625156 2843 consensus_queue.cc:1035] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20d2da3a7a24a3d991b3f929624328d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:35.634724 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.631216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:44553 (local address 127.1.47.129:50219)
0925 12:21:35.631382 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:35.631393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:35.631458 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:35.631573 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:35.631573 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:35.631574 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:35.631627 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.631628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.632510 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.632511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.633396 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:35.633398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.633908 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.633908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:35.634381 (+ 473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.634395 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.634595 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.634597 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:21:35.634631 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 12:21:35.634665 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260925 12:21:35.634732 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.631254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:50219 (local address 127.1.47.131:44553)
0925 12:21:35.631292 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.631294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.631463 (+ 169us) server_negotiation.cc:407] Connection header received
0925 12:21:35.631492 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.631493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.631504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.631572 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:35.631666 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.632420 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.633485 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.633833 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.633907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.634493 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:35.634531 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:35.634590 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.634634 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.634656 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:21:35.643255 2329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:35.643378 2329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:35.644032 2329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 863440e22cf7468a854baaa9fdef6652: Sending DeleteTablet for 3 replicas of tablet 73f0e9b81cc84bbc85ce714097f7d11c
I20260925 12:21:35.644295 2606 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:45040
I20260925 12:21:35.644312 2746 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:37750
I20260925 12:21:35.644398 2860 tablet_replica.cc:330] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: stopping tablet replica
I20260925 12:21:35.644405 2859 tablet_replica.cc:330] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: stopping tablet replica
I20260925 12:21:35.644452 2860 raft_consensus.cc:2229] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:35.644446 2466 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f0e9b81cc84bbc85ce714097f7d11c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:35 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:45364
I20260925 12:21:35.644474 2859 raft_consensus.cc:2229] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:35.644487 2860 raft_consensus.cc:2258] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.644524 2861 tablet_replica.cc:330] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: stopping tablet replica
I20260925 12:21:35.644562 2859 raft_consensus.cc:2258] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.644569 2861 raft_consensus.cc:2229] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:35.644605 2861 raft_consensus.cc:2258] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:35.644898 2861 ts_tablet_manager.cc:1905] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.644906 2860 ts_tablet_manager.cc:1905] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.644902 2859 ts_tablet_manager.cc:1905] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:35.646767 2859 ts_tablet_manager.cc:1918] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:35.646826 2859 log.cc:1196] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/73f0e9b81cc84bbc85ce714097f7d11c
I20260925 12:21:35.647035 2859 ts_tablet_manager.cc:1939] T 73f0e9b81cc84bbc85ce714097f7d11c P e0caeda5af964f0a8858cf03f77adaed: Deleting consensus metadata
I20260925 12:21:35.647140 2860 ts_tablet_manager.cc:1918] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:35.647192 2860 log.cc:1196] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/73f0e9b81cc84bbc85ce714097f7d11c
I20260925 12:21:35.647405 2860 ts_tablet_manager.cc:1939] T 73f0e9b81cc84bbc85ce714097f7d11c P c20d2da3a7a24a3d991b3f929624328d: Deleting consensus metadata
I20260925 12:21:35.647778 2314 catalog_manager.cc:4841] TS e0caeda5af964f0a8858cf03f77adaed (127.1.47.131:44553): tablet 73f0e9b81cc84bbc85ce714097f7d11c (table test-table [id=48da8a9406de435993af6f3b6dc5986d]) successfully deleted
I20260925 12:21:35.647814 2861 ts_tablet_manager.cc:1918] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:35.647881 2314 catalog_manager.cc:4841] TS c20d2da3a7a24a3d991b3f929624328d (127.1.47.130:40577): tablet 73f0e9b81cc84bbc85ce714097f7d11c (table test-table [id=48da8a9406de435993af6f3b6dc5986d]) successfully deleted
I20260925 12:21:35.647886 2861 log.cc:1196] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/73f0e9b81cc84bbc85ce714097f7d11c
I20260925 12:21:35.648102 2861 ts_tablet_manager.cc:1939] T 73f0e9b81cc84bbc85ce714097f7d11c P 554b5acaf45c4e12a00f273b5c80dba7: Deleting consensus metadata
I20260925 12:21:35.648448 2313 catalog_manager.cc:4841] TS 554b5acaf45c4e12a00f273b5c80dba7 (127.1.47.129:41313): tablet 73f0e9b81cc84bbc85ce714097f7d11c (table test-table [id=48da8a9406de435993af6f3b6dc5986d]) successfully deleted
Sep 25 12:21:35 dist-test-slave-l9ww krb5kdc[2284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:35.659614 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.648540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.1.47.129:41313)
0925 12:21:35.648582 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.648584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.648900 (+ 316us) server_negotiation.cc:407] Connection header received
0925 12:21:35.648948 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.648949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.648960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.649019 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:35.649776 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.650505 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.651625 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.651948 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.652026 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.655757 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:35.655765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:35.655766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:35.655775 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:35.658120 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.658611 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.658615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.658616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.658689 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.658930 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.658931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.658931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.659116 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:35.659257 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.659396 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.659496 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 12:21:35.660033 2526 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45380
I20260925 12:21:35.669442 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:35.661077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.1.47.190:37639)
0925 12:21:35.661123 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:35.661125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:35.661359 (+ 234us) server_negotiation.cc:407] Connection header received
0925 12:21:35.661410 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:35.661411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:35.661430 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:35.661484 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:35.662129 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.662930 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.663979 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:35.664295 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:35.664368 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:35.666082 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:35.666084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:35.666084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:35.666093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:35.668080 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.668529 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.668530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.668530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.668594 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:35.668825 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:35.668826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:35.668827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:35.668996 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:35.669095 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:35.669244 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 12:21:35.669335 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 12:21:35.669797 2329 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51526
I20260925 12:21:35.670482 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2401
I20260925 12:21:35.676636 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2541
I20260925 12:21:35.683103 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2681
I20260925 12:21:35.688411 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2297
2026-09-25T12:21:35Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2717 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:35 dist-test-slave-l9ww krb5kdc[2903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:35 dist-test-slave-l9ww krb5kdc[2903](info): set up 2 sockets
Sep 25 12:21:35 dist-test-slave-l9ww krb5kdc[2903](info): commencing operation
krb5kdc: starting...
W20260925 12:21:36.739620 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:36 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:36.863237 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46323
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:36.944159 2916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:36.944430 2916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:36.944468 2916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:36.945645 2916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:36.945717 2916 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 12:21:36.945739 2916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:36.945760 2916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:36.945787 2916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:36.945811 2916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:36.947288 2916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46323
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:36.947582 2916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:36.947821 2916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:36.952868 2922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:36.952908 2921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:36.952909 2916 server_base.cc:1034] running on GCE node
W20260925 12:21:36.952890 2924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:36.953380 2916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:36.953689 2916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:36.954854 2916 hybrid_clock.cc:648] HybridClock initialized: now 1790338896954838 us; error 37 us; skew 500 ppm
Sep 25 12:21:36 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:36.957818 2916 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:36.958331 2916 webserver.cc:460] Webserver started at http://127.1.47.190:42801/ using document root <none> and password file <none>
I20260925 12:21:36.958562 2916 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:36.958626 2916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:36.958760 2916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:36.959826 2916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "4ca0f9469bff4f9db50194bdfcd4afcc"
format_stamp: "Formatted at 2026-09-25 12:21:36 on dist-test-slave-l9ww"
server_key: "7d97289658c4607b1596c6621abc5fee"
server_key_iv: "a5275a0cbc9fadfbd89c0e6219a8cfbe"
server_key_version: "encryptionkey@0"
I20260925 12:21:36.960242 2916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "4ca0f9469bff4f9db50194bdfcd4afcc"
format_stamp: "Formatted at 2026-09-25 12:21:36 on dist-test-slave-l9ww"
server_key: "7d97289658c4607b1596c6621abc5fee"
server_key_iv: "a5275a0cbc9fadfbd89c0e6219a8cfbe"
server_key_version: "encryptionkey@0"
I20260925 12:21:36.961627 2916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:36.962325 2931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:36.962509 2916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:36.962605 2916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "4ca0f9469bff4f9db50194bdfcd4afcc"
format_stamp: "Formatted at 2026-09-25 12:21:36 on dist-test-slave-l9ww"
server_key: "7d97289658c4607b1596c6621abc5fee"
server_key_iv: "a5275a0cbc9fadfbd89c0e6219a8cfbe"
server_key_version: "encryptionkey@0"
I20260925 12:21:36.962687 2916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:36.972702 2916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:36.975045 2916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:36.975203 2916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:36.979732 2916 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46323
I20260925 12:21:36.979838 2993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46323 every 8 connection(s)
I20260925 12:21:36.980177 2916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:36.980798 2994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:36.983022 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Bootstrap starting.
I20260925 12:21:36.983603 2994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:36.983954 2994 log.cc:824] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:36.984689 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: No bootstrap required, opened a new log
I20260925 12:21:36.986752 2994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } }
I20260925 12:21:36.986891 2994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:36.986937 2994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca0f9469bff4f9db50194bdfcd4afcc, State: Initialized, Role: FOLLOWER
I20260925 12:21:36.987092 2994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } }
I20260925 12:21:36.987150 2994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:36.987177 2994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:36.987242 2994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:36.987772 2994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } }
I20260925 12:21:36.987934 2994 leader_election.cc:302] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ca0f9469bff4f9db50194bdfcd4afcc; no voters:
I20260925 12:21:36.988050 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 2916
I20260925 12:21:36.988116 2994 leader_election.cc:288] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:36.988158 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:36.988166 2997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:36.988307 2997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [term 1 LEADER]: Becoming Leader. State: Replica: 4ca0f9469bff4f9db50194bdfcd4afcc, State: Running, Role: LEADER
I20260925 12:21:36.988364 1214 external_mini_cluster.cc:1428] Setting key 57bd02bc72ee4a513fbcec48309675c4
I20260925 12:21:36.988376 2994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:36.988399 2997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } }
I20260925 12:21:36.988881 2998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } } }
I20260925 12:21:36.988974 2998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:36.989614 3007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:36.989688 2999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ca0f9469bff4f9db50194bdfcd4afcc. Latest consensus state: current_term: 1 leader_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca0f9469bff4f9db50194bdfcd4afcc" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46323 } } }
I20260925 12:21:36.989895 2999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:36.990271 3007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:36.991844 3007 catalog_manager.cc:1347] Generated new cluster ID: 162a9e817a96416bac3ac709e5902330
I20260925 12:21:36.991917 3007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:36 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:36.999311 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:36.989155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.1.47.190:46323)
0925 12:21:36.989359 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:36.989370 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:36.989405 (+ 35us) server_negotiation.cc:407] Connection header received
0925 12:21:36.989529 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:36.989536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:36.989610 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:36.989758 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:36.990667 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:36.992014 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:36.992812 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:36.993246 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:36.993361 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:36.995661 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:36.995669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:36.995669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:36.995683 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:36.997797 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:36.998181 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:36.998185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:36.998185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:36.998284 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:36.998485 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:36.998486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:36.998487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:36.998743 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:36.998915 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:36.998986 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 12:21:36.999126 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260925 12:21:37.019922 3007 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:37.020475 3007 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:37.023568 3007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Generated new TSK 0
I20260925 12:21:37.023778 3007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:37.055505 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46323
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:37.151583 3020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:37.151822 3020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:37.151871 3020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:37.153065 3020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:37.153103 3020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:37.153124 3020 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 12:21:37.153136 3020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:37.153153 3020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:37.154596 3020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:46323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:37.154817 3020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:37.155002 3020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:37.159940 3025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:37.160007 3026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:37.161381 3020 server_base.cc:1034] running on GCE node
W20260925 12:21:37.161459 3028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:37.161664 3020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:37.161886 3020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:37.163026 3020 hybrid_clock.cc:648] HybridClock initialized: now 1790338897163015 us; error 31 us; skew 500 ppm
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:37.165796 3020 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:37.166265 3020 webserver.cc:460] Webserver started at http://127.1.47.129:38425/ using document root <none> and password file <none>
I20260925 12:21:37.166498 3020 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:37.166544 3020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:37.166695 3020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:37.167752 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "89f79e14019d4e418329b359396e8aa3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "55e5343d1df076a5c68ae169b87fd08f"
server_key_iv: "e34d0654e9361a9dc6426a552277dee0"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.168175 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "89f79e14019d4e418329b359396e8aa3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "55e5343d1df076a5c68ae169b87fd08f"
server_key_iv: "e34d0654e9361a9dc6426a552277dee0"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.169540 3020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.170372 3035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.170606 3020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.170701 3020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "89f79e14019d4e418329b359396e8aa3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "55e5343d1df076a5c68ae169b87fd08f"
server_key_iv: "e34d0654e9361a9dc6426a552277dee0"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.170778 3020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:37.184463 3020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:37.186933 3020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:37.187052 3020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:37.187542 3020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:37.187582 3020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.187604 3020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:37.187639 3020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.194211 3020 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:36977
I20260925 12:21:37.194271 3150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:36977 every 8 connection(s)
I20260925 12:21:37.194640 3020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:37.198534 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.188145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:53543 (local address 127.1.47.190:46323)
0925 12:21:37.188216 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.188218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.188459 (+ 241us) server_negotiation.cc:407] Connection header received
0925 12:21:37.189065 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.189067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.189085 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.189170 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.190122 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.190995 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.192614 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.192985 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.193070 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.195723 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.195725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.195725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.195733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.197210 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.197609 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.197610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.197610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.197673 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.197914 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.197915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.197916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.198051 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.198134 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.198385 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.198464 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:37.198680 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.188143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.129:53543)
0925 12:21:37.188336 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.188379 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.188875 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.189256 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.189260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.189685 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.189988 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.189996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.191095 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.191096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.192515 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.192518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.193084 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.193085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.193242 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.193893 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.193902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.195606 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.197293 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.197307 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.197309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.197529 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.197736 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.197736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.197736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.197835 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.198129 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.198133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.198251 (+ 118us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.198344 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.198470 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260925 12:21:37.199263 3151 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46323
I20260925 12:21:37.199353 3151 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:37.199546 3151 heartbeater.cc:511] Master 127.1.47.190:46323 requested a full tablet report, sending...
I20260925 12:21:37.200220 2947 ts_manager.cc:194] Registered new tserver with Master: 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977)
I20260925 12:21:37.201046 2947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:53543
I20260925 12:21:37.201418 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3020
I20260925 12:21:37.201508 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:37.201678 1214 external_mini_cluster.cc:1428] Setting key 7fcf1e1737da5c8feca0cb439255faa5
I20260925 12:21:37.206512 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.200431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:40019 (local address 127.1.47.190:46323)
0925 12:21:37.200480 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.200482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.200628 (+ 146us) server_negotiation.cc:407] Connection header received
0925 12:21:37.200691 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.200692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.200702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.200760 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.201058 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.201799 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.202557 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.202788 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.202848 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.204021 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.204023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.204023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.204030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.205399 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.205742 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.205743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.205743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.205800 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.205988 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.205989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.205989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.206097 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.206165 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.206340 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.206423 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:21:37.206521 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.200435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.129:40019)
0925 12:21:37.200564 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.200574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.200643 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.200765 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.200765 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.200932 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.200986 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.200987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.201914 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.201916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.202475 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.202477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.202864 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.202865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.202951 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.203425 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.203431 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.203940 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.205482 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.205483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.205483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.205658 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.205863 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.205864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.205865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.205923 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.206167 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.206168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.206213 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.206297 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.206409 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:37.228660 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46323
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:37.310132 3160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:37.310351 3160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:37.310379 3160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:37.311545 3160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:37.311590 3160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:37.311610 3160 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 12:21:37.311622 3160 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:37.311640 3160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:37.313128 3160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:46323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:37.313393 3160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:37.313629 3160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:37.319016 3166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:37.319085 3168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:37.319042 3165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:37.319183 3160 server_base.cc:1034] running on GCE node
I20260925 12:21:37.319413 3160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:37.319693 3160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:37.320866 3160 hybrid_clock.cc:648] HybridClock initialized: now 1790338897320828 us; error 50 us; skew 500 ppm
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:37.323428 3160 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:37.323961 3160 webserver.cc:460] Webserver started at http://127.1.47.130:39067/ using document root <none> and password file <none>
I20260925 12:21:37.324201 3160 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:37.324266 3160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:37.324401 3160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:37.325460 3160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "29a7728a55bd44a7b213a16a2a1636e3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "5b773636cc349a52bf262858f1ded025"
server_key_iv: "009c8c1ccd4f33d641aa7c1e8905a786"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.325820 3160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "29a7728a55bd44a7b213a16a2a1636e3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "5b773636cc349a52bf262858f1ded025"
server_key_iv: "009c8c1ccd4f33d641aa7c1e8905a786"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.327099 3160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:37.327765 3175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.327968 3160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.328059 3160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "29a7728a55bd44a7b213a16a2a1636e3"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "5b773636cc349a52bf262858f1ded025"
server_key_iv: "009c8c1ccd4f33d641aa7c1e8905a786"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.328138 3160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:37.337565 3160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:37.339955 3160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:37.340111 3160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:37.340669 3160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:37.340728 3160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.340768 3160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:37.340797 3160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:37.348222 3160 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:46659
I20260925 12:21:37.348294 3290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:46659 every 8 connection(s)
I20260925 12:21:37.348623 3160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:37.350317 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.341151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:40301 (local address 127.1.47.190:46323)
0925 12:21:37.341221 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.341223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.341412 (+ 189us) server_negotiation.cc:407] Connection header received
0925 12:21:37.341880 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.341881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.341893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.341983 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.342670 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.343502 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.344698 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.344892 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.344975 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.347421 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.347423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.347423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.347432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.349000 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.349380 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.349382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.349382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.349439 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.349692 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.349693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.349693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.349809 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.349881 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.350153 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.350229 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:37.350507 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.341126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.130:40301)
0925 12:21:37.341275 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.341317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.341785 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.342065 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.342071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.342373 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.342582 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.342588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.343624 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.343625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.344582 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.344585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.344994 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.344995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.345125 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.345674 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.345702 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.347320 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.349078 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.349082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.349084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.349302 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.349510 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.349511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.349511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.349613 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.349873 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.349878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.350015 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.350124 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.350280 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260925 12:21:37.350971 3291 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46323
I20260925 12:21:37.351053 3291 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:37.351236 3291 heartbeater.cc:511] Master 127.1.47.190:46323 requested a full tablet report, sending...
I20260925 12:21:37.351930 2947 ts_manager.cc:194] Registered new tserver with Master: 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659)
I20260925 12:21:37.352320 2947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:40301
I20260925 12:21:37.353570 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3160
I20260925 12:21:37.353701 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:37.353880 1214 external_mini_cluster.cc:1428] Setting key 715d1c1ce61eb078950c0272dbf4fa0f
I20260925 12:21:37.358294 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.352027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:34447 (local address 127.1.47.190:46323)
0925 12:21:37.352065 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.352067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.352547 (+ 480us) server_negotiation.cc:407] Connection header received
0925 12:21:37.352606 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.352607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.352616 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.352712 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.353008 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.353521 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.354421 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.354607 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.354690 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.355684 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.355685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.355686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.355693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.357114 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.357488 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.357489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.357490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.357543 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.357737 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.357738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.357738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.357885 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.357958 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.358119 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.358199 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:21:37.358305 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.351962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.130:34447)
0925 12:21:37.352483 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.352494 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.352550 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.352728 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.352729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.352881 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.352942 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.352944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.353620 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.353622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.354334 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.354336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.354690 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.354691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.354764 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.355266 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.355270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.355560 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.357228 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.357230 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.357230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.357407 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.357605 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.357606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.357606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.357676 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.357950 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.357951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.358007 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.358094 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.358201 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:37.379518 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46323
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:37.461812 3300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:37.462020 3300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:37.462047 3300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:37.463217 3300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:37.463259 3300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:37.463279 3300 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 12:21:37.463291 3300 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:21:37.463339 3300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:37.464901 3300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:46323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:37.465147 3300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:37.465372 3300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:37.470292 3306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:37.470310 3308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:37.470484 3300 server_base.cc:1034] running on GCE node
W20260925 12:21:37.470533 3305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:37.470722 3300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:37.470919 3300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:37.472074 3300 hybrid_clock.cc:648] HybridClock initialized: now 1790338897472037 us; error 31 us; skew 500 ppm
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:37.474718 3300 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:37.475238 3300 webserver.cc:460] Webserver started at http://127.1.47.131:38655/ using document root <none> and password file <none>
I20260925 12:21:37.475471 3300 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:37.475535 3300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:37.475687 3300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:37.476866 3300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "5a1142025d7245bfb105b9fd6babfe81"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "f2ec1d84346bd6f5aa6e4eadbc6641fa"
server_key_iv: "2fe629c94748549982770e4e2bd454d4"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.477237 3300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "5a1142025d7245bfb105b9fd6babfe81"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "f2ec1d84346bd6f5aa6e4eadbc6641fa"
server_key_iv: "2fe629c94748549982770e4e2bd454d4"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.478597 3300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:37.479310 3315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.479477 3300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.479573 3300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "5a1142025d7245bfb105b9fd6babfe81"
format_stamp: "Formatted at 2026-09-25 12:21:37 on dist-test-slave-l9ww"
server_key: "f2ec1d84346bd6f5aa6e4eadbc6641fa"
server_key_iv: "2fe629c94748549982770e4e2bd454d4"
server_key_version: "encryptionkey@0"
I20260925 12:21:37.479660 3300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:37.487772 3300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:37.490415 3300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:37.490622 3300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:37.491146 3300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:37.491214 3300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.491250 3300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:37.491279 3300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.497339 3300 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:35721
I20260925 12:21:37.497427 3430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:35721 every 8 connection(s)
I20260925 12:21:37.497725 3300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:37.501519 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.491611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:48009 (local address 127.1.47.190:46323)
0925 12:21:37.491675 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.491678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.491927 (+ 249us) server_negotiation.cc:407] Connection header received
0925 12:21:37.492453 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.492454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.492467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.492512 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.493421 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.494237 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.495788 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.496095 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.496162 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.498908 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.498909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.498910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.498918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.500392 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.500772 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.500773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.500773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.500823 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.501022 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.501022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.501023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.501123 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.501173 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.501394 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.501457 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:21:37.501675 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.491610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.131:48009)
0925 12:21:37.491783 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.491824 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.492350 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.492633 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.492638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.493029 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.493336 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.493343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.494333 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.494334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.495688 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.495691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.496232 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.496234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.496395 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.497006 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.497014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.498805 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.500475 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.500481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.500483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.500700 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.500877 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.500877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.500877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.500971 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.501173 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.501176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.501287 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.501368 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.501485 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":111,"threads_started":1}
I20260925 12:21:37.502147 3431 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46323
I20260925 12:21:37.502230 3431 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:37.502427 3431 heartbeater.cc:511] Master 127.1.47.190:46323 requested a full tablet report, sending...
I20260925 12:21:37.502820 2948 ts_manager.cc:194] Registered new tserver with Master: 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.503347 2948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:48009
I20260925 12:21:37.504578 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3300
I20260925 12:21:37.504644 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:37.504822 1214 external_mini_cluster.cc:1428] Setting key d8c637ae1e41fcdf80446487964c6bd0
I20260925 12:21:37.505669 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:21:37.510310 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.503225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.1.47.131:57885)
0925 12:21:37.503515 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.503527 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.503585 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.504194 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.504194 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.504348 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.504423 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.504424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.505250 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.505251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.506004 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.506007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.506513 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.506514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.506588 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.507068 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.507073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.507513 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.509329 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.509329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.509330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.509490 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.509685 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.509685 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.509685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.509734 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.510027 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.510029 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.510091 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.510159 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.510238 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":76,"threads_started":1}
I20260925 12:21:37.510408 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.503281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:57885 (local address 127.1.47.190:46323)
0925 12:21:37.503318 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.503320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.503575 (+ 255us) server_negotiation.cc:407] Connection header received
0925 12:21:37.503642 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.503642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.503652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.503696 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.504510 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.505173 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.506103 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.506392 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.506435 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.507609 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.507610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.507611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.507619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.509254 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.509559 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.509560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.509560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.509622 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.509805 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.509806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.509806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.509953 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.510029 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.510207 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.510291 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:37.529491 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.522592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.1.47.190:46323)
0925 12:21:37.522727 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.522729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.522740 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:37.522764 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.522765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.522775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.522835 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.523407 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.524038 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.524780 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.524962 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.525017 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.527255 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.527256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.527256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.527263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.528582 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.528890 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.528891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.528891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.528942 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.529094 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.529094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.529094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.529193 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.529266 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.529365 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.529426 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 12:21:37.531204 2948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:37.532895 2948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:37.539016 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:35721 (local address 127.0.0.1:59182)
0925 12:21:37.535449 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.535457 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.535494 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.535820 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.535821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.535822 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:37.535879 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.535881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.537071 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.537072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.537943 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.537961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.538690 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.538767 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.538780 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.538851 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.538923 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":83,"threads_started":1}
I20260925 12:21:37.539094 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.1.47.129:36977)
0925 12:21:37.535509 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.535512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.535520 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:37.535554 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.535557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.535571 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.535805 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:37.535884 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.536587 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.537988 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.538682 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538800 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.538860 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.538954 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.538995 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":83,"threads_started":1}
I20260925 12:21:37.539377 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.1.47.131:35721)
0925 12:21:37.535521 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.535526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.535536 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:37.535574 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.535576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.535592 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.535759 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:37.535959 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.536985 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538029 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.538590 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538688 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.538749 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.539250 (+ 501us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.539290 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
I20260925 12:21:37.539463 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.1.47.130:46659)
0925 12:21:37.535456 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.535462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.535473 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:37.535515 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.535517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.535535 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.535634 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:37.535810 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.536966 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538225 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.538904 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538965 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.539012 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.539347 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.539377 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260925 12:21:37.539036 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.0.0.1:49458)
0925 12:21:37.535186 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.535229 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.535293 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.535666 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.535667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.535669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:37.535726 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.535728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.536677 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.536678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.537904 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.537907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.538816 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.538817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.538884 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.538891 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.538940 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.538972 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":58,"threads_started":1}
I20260925 12:21:37.539295 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.535093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46659 (local address 127.0.0.1:40636)
0925 12:21:37.535308 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.535319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.535370 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.535635 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.535638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.535640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:37.535718 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.535721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.537056 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.537057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.538147 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.538149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.539044 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.539045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.539120 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.539125 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.539178 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.539223 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":28,"thread_start_us":80,"threads_started":1}
I20260925 12:21:37.540349 3085 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea523383d0e445189df90ca841bd3ed (DEFAULT_TABLE table=test-table [id=5851ee16a900428289c73c2bbe29a75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.540323 3346 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea523383d0e445189df90ca841bd3ed (DEFAULT_TABLE table=test-table [id=5851ee16a900428289c73c2bbe29a75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.540417 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea523383d0e445189df90ca841bd3ed (DEFAULT_TABLE table=test-table [id=5851ee16a900428289c73c2bbe29a75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.540706 3085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea523383d0e445189df90ca841bd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.540707 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea523383d0e445189df90ca841bd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.540707 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea523383d0e445189df90ca841bd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.543655 3454 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Bootstrap starting.
I20260925 12:21:37.544449 3454 tablet_bootstrap.cc:654] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.544421 3455 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Bootstrap starting.
I20260925 12:21:37.544734 3456 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Bootstrap starting.
I20260925 12:21:37.545127 3454 log.cc:824] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:37.545403 3456 tablet_bootstrap.cc:654] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.545420 3455 tablet_bootstrap.cc:654] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.545775 3456 log.cc:824] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:37.545799 3455 log.cc:824] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:37.546052 3454 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: No bootstrap required, opened a new log
I20260925 12:21:37.546154 3454 ts_tablet_manager.cc:1397] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:21:37.546592 3456 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: No bootstrap required, opened a new log
I20260925 12:21:37.546686 3456 ts_tablet_manager.cc:1397] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:21:37.548406 3455 tablet_bootstrap.cc:492] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: No bootstrap required, opened a new log
I20260925 12:21:37.548328 3454 raft_consensus.cc:348] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.548501 3455 ts_tablet_manager.cc:1397] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 12:21:37.548517 3454 raft_consensus.cc:374] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.548550 3454 raft_consensus.cc:729] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.548699 3454 consensus_queue.cc:260] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.548681 3456 raft_consensus.cc:348] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.548851 3456 raft_consensus.cc:374] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.548892 3456 raft_consensus.cc:729] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7728a55bd44a7b213a16a2a1636e3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.549019 3454 ts_tablet_manager.cc:1428] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:37.549001 3456 consensus_queue.cc:260] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.549172 3151 heartbeater.cc:503] Master 127.1.47.190:46323 was elected leader, sending a full tablet report...
I20260925 12:21:37.549260 3456 ts_tablet_manager.cc:1428] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:37.549274 3291 heartbeater.cc:503] Master 127.1.47.190:46323 was elected leader, sending a full tablet report...
I20260925 12:21:37.550432 3455 raft_consensus.cc:348] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.550581 3455 raft_consensus.cc:374] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.550625 3455 raft_consensus.cc:729] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.550743 3455 consensus_queue.cc:260] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.551021 3455 ts_tablet_manager.cc:1428] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:21:37.551038 3431 heartbeater.cc:503] Master 127.1.47.190:46323 was elected leader, sending a full tablet report...
I20260925 12:21:37.565201 3460 raft_consensus.cc:482] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:37.565380 3460 raft_consensus.cc:504] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.565768 3460 leader_election.cc:288] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.568905 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.566004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46659 (local address 127.1.47.129:41627)
0925 12:21:37.566066 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.566078 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.566139 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.566326 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.566328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.566330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:37.566407 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.566409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.567084 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.567085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.568073 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.568076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.568651 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.568732 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.568737 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.568788 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.568837 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 12:21:37.568943 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.566078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41627 (local address 127.1.47.130:46659)
0925 12:21:37.566135 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.566137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.566148 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:37.566199 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.566199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.566212 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.566307 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:37.566478 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.567014 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568164 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.568563 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568619 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.568626 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.568815 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.568855 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 12:21:37.569078 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.566078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:60841 (local address 127.1.47.131:35721)
0925 12:21:37.566128 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.566131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.566313 (+ 182us) server_negotiation.cc:407] Connection header received
0925 12:21:37.566360 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.566361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.566375 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.566429 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:37.566591 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.567134 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568275 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.568699 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568784 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.568794 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.568958 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.569005 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:21:37.569082 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.566004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:35721 (local address 127.1.47.129:60841)
0925 12:21:37.566230 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.566241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.566304 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.566437 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.566438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.566454 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:37.566524 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.566526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.567216 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.567216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.568181 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.568183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.568809 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.568810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.568882 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.568886 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.568941 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.568985 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":17,"thread_start_us":132,"threads_started":1}
I20260925 12:21:37.569355 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea523383d0e445189df90ca841bd3ed" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3" is_pre_election: true
I20260925 12:21:37.569528 3245 raft_consensus.cc:2454] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.569525 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea523383d0e445189df90ca841bd3ed" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81" is_pre_election: true
I20260925 12:21:37.569692 3372 raft_consensus.cc:2454] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.569779 3036 leader_election.cc:302] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.569931 3460 raft_consensus.cc:2781] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:37.570003 3460 raft_consensus.cc:482] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:37.570034 3460 raft_consensus.cc:3037] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.570747 3460 raft_consensus.cc:504] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.570895 3460 leader_election.cc:288] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.571066 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea523383d0e445189df90ca841bd3ed" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3"
I20260925 12:21:37.571063 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea523383d0e445189df90ca841bd3ed" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81"
I20260925 12:21:37.571156 3245 raft_consensus.cc:3037] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.571147 3372 raft_consensus.cc:3037] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.571955 3245 raft_consensus.cc:2454] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.571970 3372 raft_consensus.cc:2454] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.572440 3039 leader_election.cc:302] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1142025d7245bfb105b9fd6babfe81, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.572569 3460 raft_consensus.cc:2781] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:37.572844 3460 raft_consensus.cc:686] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Running, Role: LEADER
I20260925 12:21:37.572962 3460 consensus_queue.cc:237] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.573904 2947 catalog_manager.cc:5495] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f79e14019d4e418329b359396e8aa3 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } health_report { overall_health: UNKNOWN } } }
W20260925 12:21:37.599200 3292 tablet.cc:2369] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 12:21:37 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:37.603766 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.596768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.1.47.190:46323)
0925 12:21:37.596839 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.596841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.597074 (+ 233us) server_negotiation.cc:407] Connection header received
0925 12:21:37.597104 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.597104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.597114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.597178 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:37.597685 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.598177 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.598915 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.599168 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.599229 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.601343 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:37.601344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:37.601344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:37.601350 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:37.602624 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.603041 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.603042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.603043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.603104 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:37.603325 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:37.603326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:37.603326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:37.603449 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:37.603511 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.603624 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.603696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 12:21:37.603925 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.596708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46323 (local address 127.0.0.1:48440)
0925 12:21:37.596928 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.596939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.597008 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.597213 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.597214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.597534 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:37.597582 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.597583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.598244 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.598245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.598846 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.598848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.599241 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.599242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.599298 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.599903 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:37.599907 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:37.601263 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:37.602703 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.602710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.602713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.602982 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.603171 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:37.603172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:37.603172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:37.603249 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:37.603509 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:37.603514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:37.603556 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.603607 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.603772 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":63,"thread_start_us":115,"threads_started":1}
I20260925 12:21:37.605902 2947 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:48440:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 12:21:37.606129 2947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:37.607649 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 3deaf5eaa5ac431d90606187f1351dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2799a10e77864db3a1e8da2a19cce362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:37.607683 3085 tablet_service.cc:1467] Processing CreateTablet for tablet 3deaf5eaa5ac431d90606187f1351dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2799a10e77864db3a1e8da2a19cce362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:37.607758 3346 tablet_service.cc:1467] Processing CreateTablet for tablet 3deaf5eaa5ac431d90606187f1351dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2799a10e77864db3a1e8da2a19cce362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:21:37.607842 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3deaf5eaa5ac431d90606187f1351dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.607878 3085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3deaf5eaa5ac431d90606187f1351dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.607920 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3deaf5eaa5ac431d90606187f1351dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.609175 3454 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: Bootstrap starting.
I20260925 12:21:37.609438 3456 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: Bootstrap starting.
I20260925 12:21:37.609644 3454 tablet_bootstrap.cc:654] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.610177 3456 tablet_bootstrap.cc:654] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.610340 3454 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: No bootstrap required, opened a new log
I20260925 12:21:37.610381 3454 ts_tablet_manager.cc:1397] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.610579 3455 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: Bootstrap starting.
I20260925 12:21:37.610551 3454 raft_consensus.cc:348] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.610698 3454 raft_consensus.cc:374] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.610734 3454 raft_consensus.cc:729] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.610785 3454 consensus_queue.cc:260] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.610854 3456 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: No bootstrap required, opened a new log
I20260925 12:21:37.610888 3454 ts_tablet_manager.cc:1428] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.610895 3456 ts_tablet_manager.cc:1397] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.611032 3460 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.611034 3456 raft_consensus.cc:348] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.611109 3460 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.611117 3456 raft_consensus.cc:374] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.611160 3456 raft_consensus.cc:729] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7728a55bd44a7b213a16a2a1636e3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.611176 3455 tablet_bootstrap.cc:654] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.611207 3456 consensus_queue.cc:260] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.611316 3456 ts_tablet_manager.cc:1428] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.611467 3461 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.611562 3461 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.611917 3455 tablet_bootstrap.cc:492] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: No bootstrap required, opened a new log
I20260925 12:21:37.611976 3455 ts_tablet_manager.cc:1397] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.612141 3455 raft_consensus.cc:348] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.612226 3455 raft_consensus.cc:374] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.612241 3455 raft_consensus.cc:729] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.612313 3455 consensus_queue.cc:260] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.612398 3455 ts_tablet_manager.cc:1428] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.612479 3462 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.612548 3462 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 12:21:37.695436 3152 tablet.cc:2369] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:21:37.748425 3432 tablet.cc:2369] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:37.802714 3460 raft_consensus.cc:482] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:37.802820 3460 raft_consensus.cc:504] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.802990 3460 leader_election.cc:288] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.803205 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3deaf5eaa5ac431d90606187f1351dec" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3" is_pre_election: true
I20260925 12:21:37.803243 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3deaf5eaa5ac431d90606187f1351dec" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81" is_pre_election: true
I20260925 12:21:37.803311 3245 raft_consensus.cc:2454] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.803331 3372 raft_consensus.cc:2454] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.803472 3039 leader_election.cc:302] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1142025d7245bfb105b9fd6babfe81, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.803591 3460 raft_consensus.cc:2781] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:37.803653 3460 raft_consensus.cc:482] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:37.803668 3460 raft_consensus.cc:3037] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.804384 3460 raft_consensus.cc:504] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.804520 3460 leader_election.cc:288] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.804692 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3deaf5eaa5ac431d90606187f1351dec" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3"
I20260925 12:21:37.804757 3245 raft_consensus.cc:3037] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.804720 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3deaf5eaa5ac431d90606187f1351dec" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81"
I20260925 12:21:37.804824 3372 raft_consensus.cc:3037] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.805364 3245 raft_consensus.cc:2454] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.805490 3372 raft_consensus.cc:2454] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.805523 3036 leader_election.cc:302] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.805645 3460 raft_consensus.cc:2781] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:37.805697 3460 raft_consensus.cc:686] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Running, Role: LEADER
I20260925 12:21:37.805742 3460 consensus_queue.cc:237] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.806005 3464 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: TxnStatusTablet state changed. Reason: New leader 89f79e14019d4e418329b359396e8aa3. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.806087 3464 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:37.806252 3476 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 12:21:37.806396 3476 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 12:21:37.806315 2948 catalog_manager.cc:5495] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f79e14019d4e418329b359396e8aa3 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:37.840245 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.837856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.1.47.129:36977)
0925 12:21:37.837906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.837908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.837916 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:37.837982 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.837983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.838003 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.838079 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:37.838210 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.838673 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.839411 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.839597 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.839694 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.839913 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:37.840007 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:37.840098 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.840140 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.840161 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:21:37.840246 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.837775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.0.0.1:49470)
0925 12:21:37.837843 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.837854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.837903 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.838082 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.838083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.838084 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:37.838138 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.838140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.838753 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.838754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.839328 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.839330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.839693 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.839694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.839760 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.839781 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:37.840070 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:37.840080 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.840128 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.840181 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 12:21:37.842129 3372 raft_consensus.cc:1261] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.842159 3245 raft_consensus.cc:1261] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.842480 3460 consensus_queue.cc:1035] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:37.842597 3464 consensus_queue.cc:1035] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:37.843760 3462 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: TxnStatusTablet state changed. Reason: New leader 89f79e14019d4e418329b359396e8aa3. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.843909 3462 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.844264 3461 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: TxnStatusTablet state changed. Reason: New leader 89f79e14019d4e418329b359396e8aa3. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.844349 3461 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.844653 3460 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.844729 3460 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:37.844805 3481 mvcc.cc:205] Tried to move back new op lower bound from 7333228125558857728 to 7333228125412708352. Current Snapshot: MvccSnapshot[applied={T|T < 7333228125558857728}]
I20260925 12:21:37.844961 3462 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.845052 3462 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 5a1142025d7245bfb105b9fd6babfe81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.846251 3464 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.846342 3464 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 89f79e14019d4e418329b359396e8aa3: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:21:37.846625 3480 mvcc.cc:205] Tried to move back new op lower bound from 7333228125558857728 to 7333228125412708352. Current Snapshot: MvccSnapshot[applied={T|T < 7333228125558857728}]
I20260925 12:21:37.846772 3461 tablet_replica.cc:439] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } } }
I20260925 12:21:37.846855 3461 tablet_replica.cc:442] T 3deaf5eaa5ac431d90606187f1351dec P 29a7728a55bd44a7b213a16a2a1636e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:21:37.850718 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.848479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49482 (local address 127.1.47.129:36977)
0925 12:21:37.848532 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.848533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.848541 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:37.848593 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.848594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.848601 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.848678 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:37.848816 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.849280 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.850057 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.850242 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.850311 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.850457 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:37.850492 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:37.850544 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.850611 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.850629 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 12:21:37.855561 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.853145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41077 (local address 127.1.47.129:36977)
0925 12:21:37.853183 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:37.853185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:37.853303 (+ 118us) server_negotiation.cc:407] Connection header received
0925 12:21:37.853363 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:37.853364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:37.853373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:37.853421 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:37.853553 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.854016 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.854797 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.854992 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.855062 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.855308 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:37.855338 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:37.855400 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:37.855468 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 12:21:37.855484 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":19}
I20260925 12:21:37.855582 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:37.853091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.1.47.129:41077)
0925 12:21:37.853245 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:37.853253 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:37.853317 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:37.853422 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:37.853422 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:37.853424 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:37.853488 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.853489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.854095 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.854096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.854718 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:37.854719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:37.855059 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:37.855060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:37.855193 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:37.855205 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:37.855402 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:37.855403 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 12:21:37.855464 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:21:37.855502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260925 12:21:37.857638 3245 raft_consensus.cc:1261] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.857652 3372 raft_consensus.cc:1261] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.857867 3460 consensus_queue.cc:1035] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:37.857951 3464 consensus_queue.cc:1035] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:37.859052 3479 mvcc.cc:205] Tried to move back new op lower bound from 7333228125623681024 to 7333228124459757568. Current Snapshot: MvccSnapshot[applied={T|T < 7333228125623681024}]
I20260925 12:21:37.871780 2947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:37.871940 2947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:37.872707 2947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Sending DeleteTablet for 3 replicas of tablet 3ea523383d0e445189df90ca841bd3ed
I20260925 12:21:37.873044 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea523383d0e445189df90ca841bd3ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:37 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40636
I20260925 12:21:37.873097 3346 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea523383d0e445189df90ca841bd3ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:37 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:59182
I20260925 12:21:37.873180 3084 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea523383d0e445189df90ca841bd3ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:37 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49458
I20260925 12:21:37.873214 2947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:37.873345 2947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:37.873438 3497 tablet_replica.cc:330] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: stopping tablet replica
I20260925 12:21:37.873729 3495 tablet_replica.cc:330] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: stopping tablet replica
I20260925 12:21:37.873809 3497 raft_consensus.cc:2229] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:37.873936 3497 raft_consensus.cc:2258] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:37.873955 3495 raft_consensus.cc:2229] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:37.874244 3495 raft_consensus.cc:2258] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:37.874398 3497 ts_tablet_manager.cc:1905] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:37.874603 3346 tablet_service.cc:1467] Processing CreateTablet for tablet b2875d7f49ef4112b5bf3feba7318aad (DEFAULT_TABLE table=test-table [id=399baac125114346847057cc16ad6db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.874630 3225 tablet_service.cc:1467] Processing CreateTablet for tablet b2875d7f49ef4112b5bf3feba7318aad (DEFAULT_TABLE table=test-table [id=399baac125114346847057cc16ad6db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.874743 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2875d7f49ef4112b5bf3feba7318aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.874744 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2875d7f49ef4112b5bf3feba7318aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.876042 3456 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Bootstrap starting.
I20260925 12:21:37.876255 3455 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Bootstrap starting.
I20260925 12:21:37.876261 3497 ts_tablet_manager.cc:1918] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:37.876324 3497 log.cc:1196] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/3ea523383d0e445189df90ca841bd3ed
I20260925 12:21:37.876478 3456 tablet_bootstrap.cc:654] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.876595 3497 ts_tablet_manager.cc:1939] T 3ea523383d0e445189df90ca841bd3ed P 89f79e14019d4e418329b359396e8aa3: Deleting consensus metadata
I20260925 12:21:37.876677 3455 tablet_bootstrap.cc:654] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.877005 3495 ts_tablet_manager.cc:1905] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:37.877048 2933 catalog_manager.cc:4841] TS 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977): tablet 3ea523383d0e445189df90ca841bd3ed (table test-table [id=5851ee16a900428289c73c2bbe29a75e]) successfully deleted
I20260925 12:21:37.877116 3084 tablet_service.cc:1467] Processing CreateTablet for tablet b2875d7f49ef4112b5bf3feba7318aad (DEFAULT_TABLE table=test-table [id=399baac125114346847057cc16ad6db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:37.877228 3084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2875d7f49ef4112b5bf3feba7318aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:37.877756 3456 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: No bootstrap required, opened a new log
I20260925 12:21:37.877807 3456 ts_tablet_manager.cc:1397] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:37.877960 3456 raft_consensus.cc:348] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.878046 3456 raft_consensus.cc:374] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.878072 3456 raft_consensus.cc:729] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7728a55bd44a7b213a16a2a1636e3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.878110 3456 consensus_queue.cc:260] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.878214 3456 ts_tablet_manager.cc:1428] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.878392 3454 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Bootstrap starting.
I20260925 12:21:37.878767 3454 tablet_bootstrap.cc:654] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:37.879323 3454 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: No bootstrap required, opened a new log
I20260925 12:21:37.879390 3454 ts_tablet_manager.cc:1397] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:37.879531 3454 raft_consensus.cc:348] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.879598 3454 raft_consensus.cc:374] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.879631 3454 raft_consensus.cc:729] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.879688 3454 consensus_queue.cc:260] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.879777 3454 ts_tablet_manager.cc:1428] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:37.880796 3495 ts_tablet_manager.cc:1918] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:37.880880 3495 log.cc:1196] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/3ea523383d0e445189df90ca841bd3ed
I20260925 12:21:37.881122 3495 ts_tablet_manager.cc:1939] T 3ea523383d0e445189df90ca841bd3ed P 29a7728a55bd44a7b213a16a2a1636e3: Deleting consensus metadata
I20260925 12:21:37.881523 2932 catalog_manager.cc:4841] TS 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659): tablet 3ea523383d0e445189df90ca841bd3ed (table test-table [id=5851ee16a900428289c73c2bbe29a75e]) successfully deleted
I20260925 12:21:37.883996 3496 tablet_replica.cc:330] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: stopping tablet replica
I20260925 12:21:37.884099 3496 raft_consensus.cc:2229] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:37.884188 3496 raft_consensus.cc:2258] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:37.884297 3455 tablet_bootstrap.cc:492] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: No bootstrap required, opened a new log
I20260925 12:21:37.884341 3455 ts_tablet_manager.cc:1397] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260925 12:21:37.884452 3455 raft_consensus.cc:348] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.884507 3455 raft_consensus.cc:374] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:37.884521 3455 raft_consensus.cc:729] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Initialized, Role: FOLLOWER
I20260925 12:21:37.884572 3455 consensus_queue.cc:260] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.884712 3455 ts_tablet_manager.cc:1428] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:37.885116 3496 ts_tablet_manager.cc:1905] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:37.886621 3496 ts_tablet_manager.cc:1918] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:37.886690 3496 log.cc:1196] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/3ea523383d0e445189df90ca841bd3ed
I20260925 12:21:37.886937 3496 ts_tablet_manager.cc:1939] T 3ea523383d0e445189df90ca841bd3ed P 5a1142025d7245bfb105b9fd6babfe81: Deleting consensus metadata
I20260925 12:21:37.887444 2935 catalog_manager.cc:4841] TS 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721): tablet 3ea523383d0e445189df90ca841bd3ed (table test-table [id=5851ee16a900428289c73c2bbe29a75e]) successfully deleted
I20260925 12:21:37.892985 3460 raft_consensus.cc:482] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:37.893080 3460 raft_consensus.cc:504] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.893281 3460 leader_election.cc:288] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.893543 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2875d7f49ef4112b5bf3feba7318aad" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3" is_pre_election: true
I20260925 12:21:37.893673 3245 raft_consensus.cc:2454] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.893653 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2875d7f49ef4112b5bf3feba7318aad" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81" is_pre_election: true
I20260925 12:21:37.893752 3372 raft_consensus.cc:2454] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 0.
I20260925 12:21:37.893949 3036 leader_election.cc:302] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.894095 3460 raft_consensus.cc:2781] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:37.894147 3460 raft_consensus.cc:482] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:37.894171 3460 raft_consensus.cc:3037] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.894897 3460 raft_consensus.cc:504] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.895062 3460 leader_election.cc:288] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659), 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721)
I20260925 12:21:37.895215 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2875d7f49ef4112b5bf3feba7318aad" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3"
I20260925 12:21:37.895291 3245 raft_consensus.cc:3037] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.895241 3372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2875d7f49ef4112b5bf3feba7318aad" candidate_uuid: "89f79e14019d4e418329b359396e8aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1142025d7245bfb105b9fd6babfe81"
I20260925 12:21:37.895326 3372 raft_consensus.cc:3037] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:37.895954 3245 raft_consensus.cc:2454] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.896142 3036 leader_election.cc:302] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:37.896235 3372 raft_consensus.cc:2454] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f79e14019d4e418329b359396e8aa3 in term 1.
I20260925 12:21:37.896260 3460 raft_consensus.cc:2781] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:37.896325 3460 raft_consensus.cc:686] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Running, Role: LEADER
I20260925 12:21:37.896385 3460 consensus_queue.cc:237] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:37.896970 2947 catalog_manager.cc:5495] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f79e14019d4e418329b359396e8aa3 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "89f79e14019d4e418329b359396e8aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:37.904661 3245 raft_consensus.cc:1261] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.904687 3372 raft_consensus.cc:1261] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Refusing update from remote peer 89f79e14019d4e418329b359396e8aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:37.904913 3464 consensus_queue.cc:1035] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:37.905042 3493 consensus_queue.cc:1035] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 12:21:37.909680 3479 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 12:21:39.428344 2946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:39.428444 2946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:39.429378 2946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Sending DeleteTablet for 3 replicas of tablet b2875d7f49ef4112b5bf3feba7318aad
I20260925 12:21:39.429697 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet b2875d7f49ef4112b5bf3feba7318aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40636
I20260925 12:21:39.429713 3346 tablet_service.cc:1514] Processing DeleteTablet for tablet b2875d7f49ef4112b5bf3feba7318aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:59182
I20260925 12:21:39.429895 3084 tablet_service.cc:1514] Processing DeleteTablet for tablet b2875d7f49ef4112b5bf3feba7318aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49458
I20260925 12:21:39.430114 3525 tablet_replica.cc:330] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: stopping tablet replica
I20260925 12:21:39.430186 3525 raft_consensus.cc:2229] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:39.430266 3525 raft_consensus.cc:2258] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.430481 3525 ts_tablet_manager.cc:1905] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:39.431959 3525 ts_tablet_manager.cc:1918] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 12:21:39.432027 3525 log.cc:1196] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/b2875d7f49ef4112b5bf3feba7318aad
I20260925 12:21:39.432278 3525 ts_tablet_manager.cc:1939] T b2875d7f49ef4112b5bf3feba7318aad P 89f79e14019d4e418329b359396e8aa3: Deleting consensus metadata
I20260925 12:21:39.432613 3523 tablet_replica.cc:330] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: stopping tablet replica
I20260925 12:21:39.432651 2933 catalog_manager.cc:4841] TS 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977): tablet b2875d7f49ef4112b5bf3feba7318aad (table test-table [id=399baac125114346847057cc16ad6db0]) successfully deleted
I20260925 12:21:39.432686 3523 raft_consensus.cc:2229] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:39.432732 3523 raft_consensus.cc:2258] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.432857 3524 tablet_replica.cc:330] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: stopping tablet replica
I20260925 12:21:39.432922 3524 raft_consensus.cc:2229] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:39.432971 3524 raft_consensus.cc:2258] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.433043 3523 ts_tablet_manager.cc:1905] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:39.433223 3524 ts_tablet_manager.cc:1905] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:39.434677 3524 ts_tablet_manager.cc:1918] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 12:21:39.434731 3524 log.cc:1196] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/b2875d7f49ef4112b5bf3feba7318aad
I20260925 12:21:39.434953 3524 ts_tablet_manager.cc:1939] T b2875d7f49ef4112b5bf3feba7318aad P 5a1142025d7245bfb105b9fd6babfe81: Deleting consensus metadata
I20260925 12:21:39.435081 3523 ts_tablet_manager.cc:1918] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 12:21:39.435166 3523 log.cc:1196] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/b2875d7f49ef4112b5bf3feba7318aad
I20260925 12:21:39.435292 2935 catalog_manager.cc:4841] TS 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721): tablet b2875d7f49ef4112b5bf3feba7318aad (table test-table [id=399baac125114346847057cc16ad6db0]) successfully deleted
I20260925 12:21:39.435465 3523 ts_tablet_manager.cc:1939] T b2875d7f49ef4112b5bf3feba7318aad P 29a7728a55bd44a7b213a16a2a1636e3: Deleting consensus metadata
I20260925 12:21:39.435793 2932 catalog_manager.cc:4841] TS 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659): tablet b2875d7f49ef4112b5bf3feba7318aad (table test-table [id=399baac125114346847057cc16ad6db0]) successfully deleted
Sep 25 12:21:39 dist-test-slave-l9ww krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 12:21:39 dist-test-slave-l9ww krb5kdc[2903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:39.454595 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.446763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.1.47.190:46323)
0925 12:21:39.446925 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.446927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.446956 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:21:39.447055 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.447055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.447068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.447156 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:39.447726 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.448303 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.449194 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.449410 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.449477 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.452065 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:39.452067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:39.452067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:39.452074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:39.453413 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:39.453800 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:39.453801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:39.453801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:39.453878 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:39.454109 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:39.454109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:39.454110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:39.454220 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:39.454300 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.454421 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.454490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260925 12:21:39.456401 2946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:39.456583 2946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:39.458105 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 (DEFAULT_TABLE table=test-table [id=6fe7a3273cfe4482a970fbcfc2657689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.458168 3084 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 (DEFAULT_TABLE table=test-table [id=6fe7a3273cfe4482a970fbcfc2657689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.458168 3346 tablet_service.cc:1467] Processing CreateTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 (DEFAULT_TABLE table=test-table [id=6fe7a3273cfe4482a970fbcfc2657689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.458262 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9e6cb3469a44879542c2beef3dd092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.458302 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9e6cb3469a44879542c2beef3dd092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.458302 3084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d9e6cb3469a44879542c2beef3dd092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.459482 3534 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Bootstrap starting.
I20260925 12:21:39.459785 3535 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Bootstrap starting.
I20260925 12:21:39.459956 3534 tablet_bootstrap.cc:654] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.460269 3535 tablet_bootstrap.cc:654] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.460286 3536 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Bootstrap starting.
I20260925 12:21:39.460695 3534 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: No bootstrap required, opened a new log
I20260925 12:21:39.460736 3534 ts_tablet_manager.cc:1397] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:39.460919 3535 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: No bootstrap required, opened a new log
I20260925 12:21:39.460968 3535 ts_tablet_manager.cc:1397] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:39.460915 3534 raft_consensus.cc:348] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.460994 3534 raft_consensus.cc:374] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.461026 3534 raft_consensus.cc:729] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7728a55bd44a7b213a16a2a1636e3, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.461066 3535 raft_consensus.cc:348] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.461078 3534 consensus_queue.cc:260] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.461199 3535 raft_consensus.cc:374] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.461236 3535 raft_consensus.cc:729] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.461258 3534 ts_tablet_manager.cc:1428] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:39.461297 3535 consensus_queue.cc:260] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.461603 3536 tablet_bootstrap.cc:654] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.461905 3535 ts_tablet_manager.cc:1428] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:39.462221 3536 tablet_bootstrap.cc:492] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: No bootstrap required, opened a new log
I20260925 12:21:39.462283 3536 ts_tablet_manager.cc:1397] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:39.462435 3536 raft_consensus.cc:348] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.462494 3536 raft_consensus.cc:374] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.462509 3536 raft_consensus.cc:729] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.462579 3536 consensus_queue.cc:260] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.462690 3536 ts_tablet_manager.cc:1428] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:39.564849 3540 raft_consensus.cc:482] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:39.564950 3540 raft_consensus.cc:504] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.565344 3540 leader_election.cc:288] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977), 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659)
I20260925 12:21:39.568464 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.565613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:56593 (local address 127.1.47.130:46659)
0925 12:21:39.565780 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.565782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.565794 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:21:39.565871 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.565872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.565884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.565958 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:39.566128 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.566651 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.567691 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.568111 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.568168 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.568192 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.568331 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.568377 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260925 12:21:39.568480 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.565580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:49649 (local address 127.1.47.129:36977)
0925 12:21:39.565735 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.565737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.565820 (+ 83us) server_negotiation.cc:407] Connection header received
0925 12:21:39.565868 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.565868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.565879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.565970 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:39.566173 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.566643 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.567676 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.568124 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.568210 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.568219 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.568375 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.568417 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260925 12:21:39.568456 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.565543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46659 (local address 127.1.47.131:56593)
0925 12:21:39.565739 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:39.565749 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:39.565809 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:39.565970 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:39.565971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:39.565974 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:39.566057 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.566059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.566752 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.566753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.567586 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.567588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.568188 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.568189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.568265 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.568269 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:39.568312 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 12:21:39.568366 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":47,"thread_start_us":109,"threads_started":1}
I20260925 12:21:39.568485 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.565529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.1.47.131:49649)
0925 12:21:39.565710 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:39.565722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:39.565812 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:39.565978 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:39.565979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:39.565981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:39.566065 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.566067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.566734 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.566735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.567579 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.567582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.568221 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.568222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.568284 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.568287 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:39.568359 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 12:21:39.568393 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":76,"thread_start_us":111,"threads_started":1}
I20260925 12:21:39.568926 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9e6cb3469a44879542c2beef3dd092" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3" is_pre_election: true
I20260925 12:21:39.569046 3245 raft_consensus.cc:2454] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 0.
I20260925 12:21:39.569000 3105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9e6cb3469a44879542c2beef3dd092" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f79e14019d4e418329b359396e8aa3" is_pre_election: true
I20260925 12:21:39.569120 3105 raft_consensus.cc:2454] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 0.
I20260925 12:21:39.569289 3317 leader_election.cc:302] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1142025d7245bfb105b9fd6babfe81, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:39.569416 3540 raft_consensus.cc:2781] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:39.569455 3540 raft_consensus.cc:482] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:39.569478 3540 raft_consensus.cc:3037] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.570170 3540 raft_consensus.cc:504] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.570320 3540 leader_election.cc:288] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 election: Requested vote from peers 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977), 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659)
I20260925 12:21:39.570479 3105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9e6cb3469a44879542c2beef3dd092" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f79e14019d4e418329b359396e8aa3"
I20260925 12:21:39.570477 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d9e6cb3469a44879542c2beef3dd092" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3"
I20260925 12:21:39.570546 3105 raft_consensus.cc:3037] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.570551 3245 raft_consensus.cc:3037] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.571265 3105 raft_consensus.cc:2454] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 1.
I20260925 12:21:39.571264 3245 raft_consensus.cc:2454] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 1.
I20260925 12:21:39.571457 3316 leader_election.cc:302] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 5a1142025d7245bfb105b9fd6babfe81; no voters:
I20260925 12:21:39.571561 3540 raft_consensus.cc:2781] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:39.571702 3540 raft_consensus.cc:686] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Running, Role: LEADER
I20260925 12:21:39.571779 3540 consensus_queue.cc:237] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.572523 2946 catalog_manager.cc:5495] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "5a1142025d7245bfb105b9fd6babfe81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } health_report { overall_health: HEALTHY } } }
I20260925 12:21:39.579747 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.577304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.1.47.131:35721)
0925 12:21:39.577453 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.577455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.577462 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:39.577518 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.577519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.577540 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.577611 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:39.577720 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.578211 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.578969 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.579180 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.579235 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.579381 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:39.579502 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:39.579559 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.579615 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.579631 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":89,"threads_started":1}
I20260925 12:21:39.584587 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.582252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:42365 (local address 127.1.47.129:36977)
0925 12:21:39.582295 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.582298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.582308 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:39.582398 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.582399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.582413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.582476 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:39.582641 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.583096 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.583888 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.584074 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.584169 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.584334 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:39.584370 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:39.584426 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.584480 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.584501 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:21:39.584610 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.582110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.1.47.131:42365)
0925 12:21:39.582257 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:39.582273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:39.582317 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:39.582477 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:39.582478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:39.582479 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:39.582548 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.582549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.583174 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.583176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.583775 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.583777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.584169 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.584170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.584237 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.584254 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:39.584427 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:39.584429 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:21:39.584477 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 12:21:39.584522 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 12:21:39.586726 3105 raft_consensus.cc:1261] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Refusing update from remote peer 5a1142025d7245bfb105b9fd6babfe81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:39.586963 3245 raft_consensus.cc:1261] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Refusing update from remote peer 5a1142025d7245bfb105b9fd6babfe81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:39.586997 3546 consensus_queue.cc:1035] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:39.587164 3546 consensus_queue.cc:1035] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:39.594542 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.592307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:49483 (local address 127.1.47.131:35721)
0925 12:21:39.592341 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:39.592343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:39.592481 (+ 138us) server_negotiation.cc:407] Connection header received
0925 12:21:39.592551 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:39.592552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:39.592563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:39.592610 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:39.592710 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.593177 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.593902 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.594111 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.594183 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.594312 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:39.594344 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:39.594386 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:39.594442 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 12:21:39.594462 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 12:21:39.594561 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:39.592292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:35721 (local address 127.1.47.129:49483)
0925 12:21:39.592411 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:39.592421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:39.592494 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:39.592611 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:39.592612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:39.592613 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:21:39.592663 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.592664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.593256 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.593257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.593822 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:39.593823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:39.594181 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:39.594182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:39.594235 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:39.594239 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:21:39.594399 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:21:39.594401 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:21:39.594441 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 12:21:39.594485 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":79,"threads_started":1}
I20260925 12:21:39.601171 2946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:39.601343 2946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:39.601974 2946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Sending DeleteTablet for 3 replicas of tablet 3d9e6cb3469a44879542c2beef3dd092
I20260925 12:21:39.602267 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40636
I20260925 12:21:39.602365 3523 tablet_replica.cc:330] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: stopping tablet replica
I20260925 12:21:39.602337 3084 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49458
I20260925 12:21:39.602427 3525 tablet_replica.cc:330] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: stopping tablet replica
I20260925 12:21:39.602412 3523 raft_consensus.cc:2229] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:39.602471 3523 raft_consensus.cc:2258] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.602478 3525 raft_consensus.cc:2229] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:39.602510 3525 raft_consensus.cc:2258] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.602708 3346 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d9e6cb3469a44879542c2beef3dd092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:39 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:59182
I20260925 12:21:39.602804 3524 tablet_replica.cc:330] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: stopping tablet replica
I20260925 12:21:39.602854 3523 ts_tablet_manager.cc:1905] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:39.602864 3524 raft_consensus.cc:2229] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:39.602959 3524 raft_consensus.cc:2258] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:39.602991 3525 ts_tablet_manager.cc:1905] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:39.604094 2946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 12:21:39.604154 3524 ts_tablet_manager.cc:1905] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:21:39.604259 2946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:39.604519 3523 ts_tablet_manager.cc:1918] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:39.604604 3523 log.cc:1196] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/3d9e6cb3469a44879542c2beef3dd092
I20260925 12:21:39.604633 3525 ts_tablet_manager.cc:1918] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:39.604683 3525 log.cc:1196] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/3d9e6cb3469a44879542c2beef3dd092
I20260925 12:21:39.604837 3523 ts_tablet_manager.cc:1939] T 3d9e6cb3469a44879542c2beef3dd092 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting consensus metadata
I20260925 12:21:39.604904 3525 ts_tablet_manager.cc:1939] T 3d9e6cb3469a44879542c2beef3dd092 P 89f79e14019d4e418329b359396e8aa3: Deleting consensus metadata
I20260925 12:21:39.605274 2932 catalog_manager.cc:4841] TS 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659): tablet 3d9e6cb3469a44879542c2beef3dd092 (table test-table [id=6fe7a3273cfe4482a970fbcfc2657689]) successfully deleted
I20260925 12:21:39.605341 2933 catalog_manager.cc:4841] TS 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977): tablet 3d9e6cb3469a44879542c2beef3dd092 (table test-table [id=6fe7a3273cfe4482a970fbcfc2657689]) successfully deleted
I20260925 12:21:39.605459 3225 tablet_service.cc:1467] Processing CreateTablet for tablet c00dfd14a4db477cb949820076cc2a50 (DEFAULT_TABLE table=test-table [id=6ac6b35118ca4233bddaaac707f328af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.605583 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00dfd14a4db477cb949820076cc2a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.605580 3346 tablet_service.cc:1467] Processing CreateTablet for tablet c00dfd14a4db477cb949820076cc2a50 (DEFAULT_TABLE table=test-table [id=6ac6b35118ca4233bddaaac707f328af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.605737 3346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00dfd14a4db477cb949820076cc2a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.606384 3534 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Bootstrap starting.
I20260925 12:21:39.606814 3535 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Bootstrap starting.
I20260925 12:21:39.605742 3524 ts_tablet_manager.cc:1918] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:21:39.606963 3524 log.cc:1196] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/3d9e6cb3469a44879542c2beef3dd092
I20260925 12:21:39.606972 3534 tablet_bootstrap.cc:654] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.607198 3535 tablet_bootstrap.cc:654] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.607213 3524 ts_tablet_manager.cc:1939] T 3d9e6cb3469a44879542c2beef3dd092 P 5a1142025d7245bfb105b9fd6babfe81: Deleting consensus metadata
I20260925 12:21:39.607496 2935 catalog_manager.cc:4841] TS 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721): tablet 3d9e6cb3469a44879542c2beef3dd092 (table test-table [id=6fe7a3273cfe4482a970fbcfc2657689]) successfully deleted
I20260925 12:21:39.607671 3084 tablet_service.cc:1467] Processing CreateTablet for tablet c00dfd14a4db477cb949820076cc2a50 (DEFAULT_TABLE table=test-table [id=6ac6b35118ca4233bddaaac707f328af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:39.607764 3084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00dfd14a4db477cb949820076cc2a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:39.607896 3535 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: No bootstrap required, opened a new log
I20260925 12:21:39.607935 3535 ts_tablet_manager.cc:1397] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:39.607997 3534 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: No bootstrap required, opened a new log
I20260925 12:21:39.608042 3534 ts_tablet_manager.cc:1397] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:21:39.608101 3535 raft_consensus.cc:348] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.608160 3535 raft_consensus.cc:374] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.608178 3535 raft_consensus.cc:729] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.608179 3534 raft_consensus.cc:348] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.608219 3535 consensus_queue.cc:260] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.608242 3534 raft_consensus.cc:374] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.608263 3534 raft_consensus.cc:729] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7728a55bd44a7b213a16a2a1636e3, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.608318 3535 ts_tablet_manager.cc:1428] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:39.608307 3534 consensus_queue.cc:260] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.608397 3534 ts_tablet_manager.cc:1428] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:39.609071 3536 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3: Bootstrap starting.
I20260925 12:21:39.609382 3536 tablet_bootstrap.cc:654] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:39.609992 3536 tablet_bootstrap.cc:492] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3: No bootstrap required, opened a new log
I20260925 12:21:39.610052 3536 ts_tablet_manager.cc:1397] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:39.610158 3536 raft_consensus.cc:348] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.610234 3536 raft_consensus.cc:374] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:39.610257 3536 raft_consensus.cc:729] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f79e14019d4e418329b359396e8aa3, State: Initialized, Role: FOLLOWER
I20260925 12:21:39.610302 3536 consensus_queue.cc:260] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.610391 3536 ts_tablet_manager.cc:1428] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:39.692788 3546 raft_consensus.cc:482] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:39.692906 3546 raft_consensus.cc:504] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.693110 3546 leader_election.cc:288] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977), 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659)
I20260925 12:21:39.693295 3105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00dfd14a4db477cb949820076cc2a50" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f79e14019d4e418329b359396e8aa3" is_pre_election: true
I20260925 12:21:39.693363 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00dfd14a4db477cb949820076cc2a50" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3" is_pre_election: true
I20260925 12:21:39.693410 3105 raft_consensus.cc:2454] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 0.
I20260925 12:21:39.693445 3245 raft_consensus.cc:2454] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 0.
I20260925 12:21:39.693568 3317 leader_election.cc:302] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1142025d7245bfb105b9fd6babfe81, 89f79e14019d4e418329b359396e8aa3; no voters:
I20260925 12:21:39.693674 3546 raft_consensus.cc:2781] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:39.693708 3546 raft_consensus.cc:482] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:39.693728 3546 raft_consensus.cc:3037] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.694435 3546 raft_consensus.cc:504] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.694564 3546 leader_election.cc:288] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 election: Requested vote from peers 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977), 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659)
I20260925 12:21:39.694726 3245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00dfd14a4db477cb949820076cc2a50" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7728a55bd44a7b213a16a2a1636e3"
I20260925 12:21:39.694718 3105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00dfd14a4db477cb949820076cc2a50" candidate_uuid: "5a1142025d7245bfb105b9fd6babfe81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f79e14019d4e418329b359396e8aa3"
I20260925 12:21:39.694793 3105 raft_consensus.cc:3037] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.694793 3245 raft_consensus.cc:3037] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:39.695516 3245 raft_consensus.cc:2454] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 1.
I20260925 12:21:39.695516 3105 raft_consensus.cc:2454] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1142025d7245bfb105b9fd6babfe81 in term 1.
I20260925 12:21:39.695693 3316 leader_election.cc:302] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a7728a55bd44a7b213a16a2a1636e3, 5a1142025d7245bfb105b9fd6babfe81; no voters:
I20260925 12:21:39.695775 3546 raft_consensus.cc:2781] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:39.695817 3546 raft_consensus.cc:686] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1142025d7245bfb105b9fd6babfe81, State: Running, Role: LEADER
I20260925 12:21:39.695897 3546 consensus_queue.cc:237] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } }
I20260925 12:21:39.696530 2946 catalog_manager.cc:5495] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "5a1142025d7245bfb105b9fd6babfe81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1142025d7245bfb105b9fd6babfe81" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35721 } health_report { overall_health: HEALTHY } } }
I20260925 12:21:39.704277 3245 raft_consensus.cc:1261] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Refusing update from remote peer 5a1142025d7245bfb105b9fd6babfe81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:39.704311 3105 raft_consensus.cc:1261] T c00dfd14a4db477cb949820076cc2a50 P 89f79e14019d4e418329b359396e8aa3 [term 1 FOLLOWER]: Refusing update from remote peer 5a1142025d7245bfb105b9fd6babfe81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:39.704488 3546 consensus_queue.cc:1035] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7728a55bd44a7b213a16a2a1636e3" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:39.704612 3555 consensus_queue.cc:1035] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f79e14019d4e418329b359396e8aa3" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 12:21:39.708848 3507 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 12:21:41.226250 2947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:41.226361 2947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:41.227348 2947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ca0f9469bff4f9db50194bdfcd4afcc: Sending DeleteTablet for 3 replicas of tablet c00dfd14a4db477cb949820076cc2a50
I20260925 12:21:41.227671 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet c00dfd14a4db477cb949820076cc2a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:41 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40636
I20260925 12:21:41.227685 3084 tablet_service.cc:1514] Processing DeleteTablet for tablet c00dfd14a4db477cb949820076cc2a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:41 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49458
I20260925 12:21:41.227792 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3020
I20260925 12:21:41.227916 3346 tablet_service.cc:1514] Processing DeleteTablet for tablet c00dfd14a4db477cb949820076cc2a50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:41 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:59182
I20260925 12:21:41.229346 3589 tablet_replica.cc:330] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: stopping tablet replica
I20260925 12:21:41.229416 3589 raft_consensus.cc:2229] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:41.229486 3589 raft_consensus.cc:2258] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:41.229604 3588 tablet_replica.cc:330] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: stopping tablet replica
I20260925 12:21:41.229658 3588 raft_consensus.cc:2229] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:41.229701 3588 raft_consensus.cc:2258] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:41.229964 3589 ts_tablet_manager.cc:1905] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:41.229984 3588 ts_tablet_manager.cc:1905] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:41.231416 3588 ts_tablet_manager.cc:1918] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 12:21:41.231469 3588 log.cc:1196] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/c00dfd14a4db477cb949820076cc2a50
I20260925 12:21:41.231632 3588 ts_tablet_manager.cc:1939] T c00dfd14a4db477cb949820076cc2a50 P 29a7728a55bd44a7b213a16a2a1636e3: Deleting consensus metadata
I20260925 12:21:41.232160 2932 catalog_manager.cc:4841] TS 29a7728a55bd44a7b213a16a2a1636e3 (127.1.47.130:46659): tablet c00dfd14a4db477cb949820076cc2a50 (table test-table [id=6ac6b35118ca4233bddaaac707f328af]) successfully deleted
I20260925 12:21:41.232210 3589 ts_tablet_manager.cc:1918] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 12:21:41.232266 3589 log.cc:1196] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/c00dfd14a4db477cb949820076cc2a50
I20260925 12:21:41.232501 3589 ts_tablet_manager.cc:1939] T c00dfd14a4db477cb949820076cc2a50 P 5a1142025d7245bfb105b9fd6babfe81: Deleting consensus metadata
I20260925 12:21:41.232923 2935 catalog_manager.cc:4841] TS 5a1142025d7245bfb105b9fd6babfe81 (127.1.47.131:35721): tablet c00dfd14a4db477cb949820076cc2a50 (table test-table [id=6ac6b35118ca4233bddaaac707f328af]) successfully deleted
W20260925 12:21:41.235688 2933 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:36977 (error 108)
I20260925 12:21:41.236045 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3160
I20260925 12:21:41.237160 3591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:41.236211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36977 (local address 127.0.0.1:52942)
0925 12:21:41.237077 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:41.237124 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":82,"threads_started":1}
W20260925 12:21:41.237278 2933 catalog_manager.cc:4584] TS 89f79e14019d4e418329b359396e8aa3 (127.1.47.129:36977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c00dfd14a4db477cb949820076cc2a50: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36977: connect: Connection refused (error 111)
I20260925 12:21:41.241088 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3300
I20260925 12:21:41.246785 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 2916
2026-09-25T12:21:41Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5557 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:41 dist-test-slave-l9ww krb5kdc[3594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:41 dist-test-slave-l9ww krb5kdc[3594](info): set up 2 sockets
Sep 25 12:21:41 dist-test-slave-l9ww krb5kdc[3594](info): commencing operation
krb5kdc: starting...
W20260925 12:21:42.319653 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:42.443742 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42391
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:42.526266 3607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:42.526535 3607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:42.526572 3607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:42.527922 3607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:42.528026 3607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:42.528056 3607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:42.528080 3607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:42.528113 3607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:42.530009 3607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:42391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:42.530339 3607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:42.530594 3607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:42.537127 3615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:42.537181 3607 server_base.cc:1034] running on GCE node
W20260925 12:21:42.537248 3613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:42.537416 3612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:42.537665 3607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:42.537966 3607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:42.539120 3607 hybrid_clock.cc:648] HybridClock initialized: now 1790338902539093 us; error 30 us; skew 500 ppm
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:42.541625 3607 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:42.542083 3607 webserver.cc:460] Webserver started at http://127.1.47.190:41497/ using document root <none> and password file <none>
I20260925 12:21:42.542305 3607 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:42.542371 3607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:42.542513 3607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:42.543591 3607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "393f43004584444c817eee175986393a"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "d1da9beaeb79cd56f8f46d66b98fb775"
server_key_iv: "1ca7e2c61772831e2e665e8ec1880843"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.544009 3607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "393f43004584444c817eee175986393a"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "d1da9beaeb79cd56f8f46d66b98fb775"
server_key_iv: "1ca7e2c61772831e2e665e8ec1880843"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.545360 3607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:42.546078 3622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.546252 3607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:42.546345 3607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "393f43004584444c817eee175986393a"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "d1da9beaeb79cd56f8f46d66b98fb775"
server_key_iv: "1ca7e2c61772831e2e665e8ec1880843"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.546428 3607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:42.564980 3607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:42.567157 3607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:42.567330 3607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:42.571270 3607 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:42391
I20260925 12:21:42.571313 3674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:42391 every 8 connection(s)
I20260925 12:21:42.571699 3607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:42.572613 3675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:42.574884 3675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a: Bootstrap starting.
I20260925 12:21:42.575438 3675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:42.575680 3675 log.cc:824] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:42.576426 3675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a: No bootstrap required, opened a new log
I20260925 12:21:42.578088 3675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } }
I20260925 12:21:42.578251 3675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:42.578277 3675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393f43004584444c817eee175986393a, State: Initialized, Role: FOLLOWER
I20260925 12:21:42.578440 3675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } }
I20260925 12:21:42.578531 3675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:42.578557 3675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:42.578588 3675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:42.579128 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3607
I20260925 12:21:42.579136 3675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } }
I20260925 12:21:42.579226 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:42.579258 3675 leader_election.cc:302] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 393f43004584444c817eee175986393a; no voters:
I20260925 12:21:42.579424 1214 external_mini_cluster.cc:1428] Setting key fbf0b1c0c153e77cd2de474c93a59d5f
I20260925 12:21:42.579488 3675 leader_election.cc:288] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:42.579524 3678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:42.579779 3675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:42.579988 3678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [term 1 LEADER]: Becoming Leader. State: Replica: 393f43004584444c817eee175986393a, State: Running, Role: LEADER
I20260925 12:21:42.580143 3678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } }
I20260925 12:21:42.580560 3679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "393f43004584444c817eee175986393a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } } }
I20260925 12:21:42.580651 3679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:42.580826 3681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 393f43004584444c817eee175986393a. Latest consensus state: current_term: 1 leader_uuid: "393f43004584444c817eee175986393a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393f43004584444c817eee175986393a" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42391 } } }
I20260925 12:21:42.580893 3681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:42.580891 3689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:42.581465 3689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:42.583351 3689 catalog_manager.cc:1347] Generated new cluster ID: 01e8c982df3b4876b4d3f8eec548f3bf
I20260925 12:21:42.583405 3689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:42.590373 3689 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:42.590899 3689 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:42.591773 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:42.579977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.1.47.190:42391)
0925 12:21:42.580214 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:42.580224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:42.580423 (+ 199us) server_negotiation.cc:407] Connection header received
0925 12:21:42.580669 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:42.580678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:42.580740 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:42.580876 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:42.581821 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.583069 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.584148 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.584766 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.584875 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:42.587692 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:42.587697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:42.587697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:42.587717 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:42.589876 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.590291 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.590294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.590295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.590408 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.590637 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.590638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.590638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.590957 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:42.591095 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:42.591362 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 12:21:42.591527 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":190,"thread_start_us":65,"threads_started":1}
I20260925 12:21:42.601244 3689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 393f43004584444c817eee175986393a: Generated new TSK 0
I20260925 12:21:42.601464 3689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:42.631108 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:42.715840 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:42.716080 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:42.716105 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:42.717321 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:42.717366 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:42.717394 3701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:42.718833 3701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:42.719090 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:42.719347 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:42.724409 3707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:42.724432 3706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:42.724409 3709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:42.724720 3701 server_base.cc:1034] running on GCE node
I20260925 12:21:42.724968 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:42.725230 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:42.726390 3701 hybrid_clock.cc:648] HybridClock initialized: now 1790338902726376 us; error 37 us; skew 500 ppm
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:42.729133 3701 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:42.729655 3701 webserver.cc:460] Webserver started at http://127.1.47.129:44107/ using document root <none> and password file <none>
I20260925 12:21:42.729916 3701 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:42.729992 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:42.730145 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:42.731269 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "eea1b98026184d2da2e31991f337e3c0"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "13fbea28b2ed60435a2e4a582edb98ec"
server_key_iv: "29accf5e8dfd5609f787c4c70148247f"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.731660 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "eea1b98026184d2da2e31991f337e3c0"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "13fbea28b2ed60435a2e4a582edb98ec"
server_key_iv: "29accf5e8dfd5609f787c4c70148247f"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.733276 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:42.734115 3716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.734284 3701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:42.734393 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "eea1b98026184d2da2e31991f337e3c0"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "13fbea28b2ed60435a2e4a582edb98ec"
server_key_iv: "29accf5e8dfd5609f787c4c70148247f"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.734473 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:42.754670 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:42.757148 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:42.757339 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:42.757594 3701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:42.757961 3701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:42.758020 3701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.758059 3701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:42.758087 3701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.764200 3701 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43499
I20260925 12:21:42.764254 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43499 every 8 connection(s)
I20260925 12:21:42.764637 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:42.766249 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3701
I20260925 12:21:42.766366 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:42.766530 1214 external_mini_cluster.cc:1428] Setting key 39d1c00298c74a697004607204f1b2c6
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
I20260925 12:21:42.775642 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:42.765501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41023 (local address 127.1.47.190:42391)
0925 12:21:42.765628 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:42.765631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:42.765796 (+ 165us) server_negotiation.cc:407] Connection header received
0925 12:21:42.766359 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:42.766360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:42.766373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:42.766431 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:42.767459 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.768035 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.769578 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.769778 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.769838 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:42.772505 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:42.772507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:42.772507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:42.772516 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:42.774066 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.774565 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.774566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.774567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.774631 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.774898 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.774899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.774900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.775023 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:42.775145 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:42.775444 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 12:21:42.775541 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 12:21:42.775895 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:42.765451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42391 (local address 127.1.47.129:41023)
0925 12:21:42.765665 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:42.765710 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:42.766221 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:42.766520 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:42.766523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:42.767152 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:42.767360 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:42.767365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.768121 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.768123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:42.769495 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:42.769498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.769861 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.769862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:42.770024 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:42.770643 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:42.770654 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:42.772399 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:42.774196 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:42.774203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:42.774206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:42.774489 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:42.774703 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:42.774704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:42.774705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:42.774828 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:42.775139 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:42.775146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:42.775300 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 12:21:42.775415 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 12:21:42.775588 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
I20260925 12:21:42.776455 3830 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42391
I20260925 12:21:42.776571 3830 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
I20260925 12:21:42.776762 3830 heartbeater.cc:511] Master 127.1.47.190:42391 requested a full tablet report, sending...
I20260925 12:21:42.777379 3639 ts_manager.cc:194] Registered new tserver with Master: eea1b98026184d2da2e31991f337e3c0 (127.1.47.129:43499)
I20260925 12:21:42.778272 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:41023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:42.792479 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:42.878191 3835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:42.878518 3835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:42.878819 3835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:42.880502 3835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:42.880570 3835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:42.880623 3835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:42.882229 3835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:42.882514 3835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:42.882761 3835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:42.887758 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:42.887771 3843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:42.887773 3841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:42.888015 3835 server_base.cc:1034] running on GCE node
I20260925 12:21:42.888299 3835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:42.888540 3835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:42.889714 3835 hybrid_clock.cc:648] HybridClock initialized: now 1790338902889695 us; error 49 us; skew 500 ppm
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:42.892248 3835 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:42.892699 3835 webserver.cc:460] Webserver started at http://127.1.47.130:34413/ using document root <none> and password file <none>
I20260925 12:21:42.892928 3835 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:42.892973 3835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:42.893115 3835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:42.894181 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "95c478171a734a0999e68f1095ccf4c3"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "9180e867c5c8aaa89615e734011b4f06"
server_key_iv: "af722011ed3a5253ea47b01a0829cf66"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.894547 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "95c478171a734a0999e68f1095ccf4c3"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "9180e867c5c8aaa89615e734011b4f06"
server_key_iv: "af722011ed3a5253ea47b01a0829cf66"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.895965 3835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 12:21:42.896790 3850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.897029 3835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:42.897135 3835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "95c478171a734a0999e68f1095ccf4c3"
format_stamp: "Formatted at 2026-09-25 12:21:42 on dist-test-slave-l9ww"
server_key: "9180e867c5c8aaa89615e734011b4f06"
server_key_iv: "af722011ed3a5253ea47b01a0829cf66"
server_key_version: "encryptionkey@0"
I20260925 12:21:42.897215 3835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:42.912904 3835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:42.914896 3835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:42.915041 3835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:42.915287 3835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:42.915606 3835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:42.915639 3835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.915663 3835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:42.915705 3835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:42.922195 3835 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:38757
I20260925 12:21:42.922231 3963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:38757 every 8 connection(s)
I20260925 12:21:42.922605 3835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:42.927585 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3835
I20260925 12:21:42.927706 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:42.927969 1214 external_mini_cluster.cc:1428] Setting key bbaac24defe28082bc3fcd1e2b31652c
Sep 25 12:21:42 dist-test-slave-l9ww krb5kdc[3594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:42.933288 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:42.923418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:36585 (local address 127.1.47.190:42391)
0925 12:21:42.923461 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:42.923467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:42.923690 (+ 223us) server_negotiation.cc:407] Connection header received
0925 12:21:42.924249 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:42.924250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:42.924262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:42.924336 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:42.925358 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.925894 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.926989 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.927202 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.927270 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:42.929915 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:42.929917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:42.929918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:42.929926 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:42.931534 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.932118 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.932119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.932119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.932188 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:42.932562 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:42.932563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:42.932563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:42.932683 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:42.932741 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:42.933127 (+ 386us) server_negotiation.cc:299] Negotiation successful
0925 12:21:42.933196 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:42.933647 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:42.923384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42391 (local address 127.1.47.130:36585)
0925 12:21:42.923550 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:42.923594 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:42.924159 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:42.924398 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:42.924401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:42.925010 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:42.925214 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:42.925220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.925985 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.925986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:42.926870 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:42.926872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:42.927297 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:42.927298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:42.927449 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:42.928114 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:42.928125 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:42.929811 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:42.931621 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:42.931628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:42.931631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:42.932033 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:42.932281 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:42.932282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:42.932283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:42.932449 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:42.932758 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:42.932765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:42.932963 (+ 198us) client_negotiation.cc:769] Sending connection context
0925 12:21:42.933099 (+ 136us) client_negotiation.cc:240] Negotiation successful
0925 12:21:42.933323 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260925 12:21:42.934182 3964 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42391
I20260925 12:21:42.934306 3964 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:42.934532 3964 heartbeater.cc:511] Master 127.1.47.190:42391 requested a full tablet report, sending...
I20260925 12:21:42.934966 3639 ts_manager.cc:194] Registered new tserver with Master: 95c478171a734a0999e68f1095ccf4c3 (127.1.47.130:38757)
I20260925 12:21:42.935562 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:36585
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:42.955325 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:43.036593 3969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:43.036844 3969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:43.036870 3969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:43.038132 3969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:43.038185 3969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:43.038215 3969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:43.039789 3969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:43.040104 3969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:43.040365 3969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:43.045629 3974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:43.045698 3975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:43.045764 3977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:43.046135 3969 server_base.cc:1034] running on GCE node
I20260925 12:21:43.046401 3969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:43.046684 3969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:43.047894 3969 hybrid_clock.cc:648] HybridClock initialized: now 1790338903047828 us; error 39 us; skew 500 ppm
Sep 25 12:21:43 dist-test-slave-l9ww krb5kdc[3594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:43.050674 3969 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:43.051242 3969 webserver.cc:460] Webserver started at http://127.1.47.131:35127/ using document root <none> and password file <none>
I20260925 12:21:43.051496 3969 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:43.051565 3969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:43.051712 3969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:43.052891 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "ff98c84875b846ef9652367f08b93905"
format_stamp: "Formatted at 2026-09-25 12:21:43 on dist-test-slave-l9ww"
server_key: "e4699862c897959716e4f6c6f57b797d"
server_key_iv: "301a8a6f28b8f28d7e261ba225a281d9"
server_key_version: "encryptionkey@0"
I20260925 12:21:43.053267 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "ff98c84875b846ef9652367f08b93905"
format_stamp: "Formatted at 2026-09-25 12:21:43 on dist-test-slave-l9ww"
server_key: "e4699862c897959716e4f6c6f57b797d"
server_key_iv: "301a8a6f28b8f28d7e261ba225a281d9"
server_key_version: "encryptionkey@0"
I20260925 12:21:43.054636 3969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:43.055368 3984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:43.055563 3969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:43.055670 3969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "ff98c84875b846ef9652367f08b93905"
format_stamp: "Formatted at 2026-09-25 12:21:43 on dist-test-slave-l9ww"
server_key: "e4699862c897959716e4f6c6f57b797d"
server_key_iv: "301a8a6f28b8f28d7e261ba225a281d9"
server_key_version: "encryptionkey@0"
I20260925 12:21:43.055752 3969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:43.077646 3969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:43.080255 3969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:43.080446 3969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:43.080698 3969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:43.081073 3969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:43.081128 3969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:43.081188 3969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:43.081215 3969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:43.087016 3969 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:46053
I20260925 12:21:43.087052 4097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:46053 every 8 connection(s)
I20260925 12:21:43.087379 3969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:43.090330 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 3969
I20260925 12:21:43.090430 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:43.090595 1214 external_mini_cluster.cc:1428] Setting key ce43b248e2bdbfbd3ccedcecdf515357
Sep 25 12:21:43 dist-test-slave-l9ww krb5kdc[3594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:43.097043 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:43.088117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:53565 (local address 127.1.47.190:42391)
0925 12:21:43.088177 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:43.088179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:43.088389 (+ 210us) server_negotiation.cc:407] Connection header received
0925 12:21:43.088923 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:43.088925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:43.088937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:43.088995 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:43.089946 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:43.090690 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:43.091806 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:43.092054 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:43.092139 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:43.094300 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:43.094302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:43.094302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:43.094309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:43.095552 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:43.096031 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:43.096032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:43.096033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:43.096093 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:43.096377 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:43.096378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:43.096378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:43.096498 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:43.096565 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:43.096856 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 12:21:43.096947 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:21:43.097252 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:43.088123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42391 (local address 127.1.47.131:53565)
0925 12:21:43.088269 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:43.088304 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:43.088816 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:43.089071 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:43.089076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:43.089651 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:43.089847 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:43.089851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:43.090794 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:43.090795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:43.091700 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:43.091702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:43.092147 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:43.092148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:43.092311 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:43.092830 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:43.092838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:43.094185 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:43.095636 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:43.095642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:43.095645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:43.095943 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:43.096161 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:43.096162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:43.096162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:43.096303 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:43.096566 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:43.096573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:43.096718 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 12:21:43.096837 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 12:21:43.097001 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260925 12:21:43.097770 4098 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42391
I20260925 12:21:43.097875 4098 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:43.098059 4098 heartbeater.cc:511] Master 127.1.47.190:42391 requested a full tablet report, sending...
I20260925 12:21:43.098481 3639 ts_manager.cc:194] Registered new tserver with Master: ff98c84875b846ef9652367f08b93905 (127.1.47.131:46053)
I20260925 12:21:43.098897 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:53565
I20260925 12:21:43.102212 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 12:21:43.113634 3682 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:43.112596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.1.47.190:42391)
0925 12:21:43.112690 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:43.112692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:43.112796 (+ 104us) server_negotiation.cc:407] Connection header received
0925 12:21:43.112914 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:43.112915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:43.112927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:43.112996 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:43.113577 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49324: BlockingRecv error: recv got EOF from 127.0.0.1:49324 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 12:21:43.114550 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3701
I20260925 12:21:43.119522 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3835
I20260925 12:21:43.123778 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3969
I20260925 12:21:43.128500 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 3607
2026-09-25T12:21:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1880 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:43 dist-test-slave-l9ww krb5kdc[4110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:43 dist-test-slave-l9ww krb5kdc[4110](info): set up 2 sockets
Sep 25 12:21:43 dist-test-slave-l9ww krb5kdc[4110](info): commencing operation
krb5kdc: starting...
W20260925 12:21:44.159812 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:44.283581 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40273
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:40273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:44.364943 4123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:44.365242 4123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:44.365288 4123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:44.366528 4123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:44.366608 4123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:44.366633 4123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:44.366657 4123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:44.366684 4123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:44.368311 4123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:40273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:44.368602 4123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:44.368845 4123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:44.374363 4131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:44.374348 4129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:44.374348 4128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:44.374485 4123 server_base.cc:1034] running on GCE node
I20260925 12:21:44.374851 4123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:44.375187 4123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:44.376353 4123 hybrid_clock.cc:648] HybridClock initialized: now 1790338904376346 us; error 36 us; skew 500 ppm
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:44.378841 4123 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:44.379361 4123 webserver.cc:460] Webserver started at http://127.1.47.190:41377/ using document root <none> and password file <none>
I20260925 12:21:44.379608 4123 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:44.379674 4123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:44.379812 4123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:44.380923 4123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec"
format_stamp: "Formatted at 2026-09-25 12:21:44 on dist-test-slave-l9ww"
server_key: "ea45926d493a3b35bd4e0726b823d1d8"
server_key_iv: "c8591c2e46520f958c071e1242e50f0b"
server_key_version: "encryptionkey@0"
I20260925 12:21:44.381284 4123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec"
format_stamp: "Formatted at 2026-09-25 12:21:44 on dist-test-slave-l9ww"
server_key: "ea45926d493a3b35bd4e0726b823d1d8"
server_key_iv: "c8591c2e46520f958c071e1242e50f0b"
server_key_version: "encryptionkey@0"
I20260925 12:21:44.382660 4123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:44.383505 4138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:44.383733 4123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:44.383893 4123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec"
format_stamp: "Formatted at 2026-09-25 12:21:44 on dist-test-slave-l9ww"
server_key: "ea45926d493a3b35bd4e0726b823d1d8"
server_key_iv: "c8591c2e46520f958c071e1242e50f0b"
server_key_version: "encryptionkey@0"
I20260925 12:21:44.384025 4123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:44.403242 4123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:44.405598 4123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:44.405798 4123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:44.410058 4123 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:40273
I20260925 12:21:44.410099 4190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:40273 every 8 connection(s)
I20260925 12:21:44.410483 4123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:44.411365 4191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:44.413707 4191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: Bootstrap starting.
I20260925 12:21:44.414325 4191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:44.414628 4191 log.cc:824] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:44.415371 4191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: No bootstrap required, opened a new log
I20260925 12:21:44.417045 4191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } }
I20260925 12:21:44.417194 4191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:44.417218 4191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b5a068c38b4ec08e6dc6e9c033e9ec, State: Initialized, Role: FOLLOWER
I20260925 12:21:44.417390 4191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } }
I20260925 12:21:44.417454 4191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:44.417475 4191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:44.417505 4191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:44.418672 4191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } }
I20260925 12:21:44.418764 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4123
I20260925 12:21:44.418802 4191 leader_election.cc:302] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b5a068c38b4ec08e6dc6e9c033e9ec; no voters:
I20260925 12:21:44.418855 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:44.418982 4191 leader_election.cc:288] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:44.419056 1214 external_mini_cluster.cc:1428] Setting key c06fb8476310111f97642d0c9209fbf2
I20260925 12:21:44.419106 4194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:44.419263 4191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:44.419305 4194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [term 1 LEADER]: Becoming Leader. State: Replica: 53b5a068c38b4ec08e6dc6e9c033e9ec, State: Running, Role: LEADER
I20260925 12:21:44.419507 4194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } }
I20260925 12:21:44.420063 4194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } } }
I20260925 12:21:44.420274 4196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b5a068c38b4ec08e6dc6e9c033e9ec. Latest consensus state: current_term: 1 leader_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5a068c38b4ec08e6dc6e9c033e9ec" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40273 } } }
I20260925 12:21:44.420272 4194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:44.420425 4196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec [sys.catalog]: This master's current role is: LEADER
W20260925 12:21:44.421075 4211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:44.421170 4211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:44.421368 4210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:44.421577 4210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:44.424199 4210 catalog_manager.cc:1347] Generated new cluster ID: 5bd6f5d1671d4e8fbc79204319c69922
I20260925 12:21:44.424249 4210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:44.429513 4210 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:44.430172 4210 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:44.433334 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:44.419898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.1.47.190:40273)
0925 12:21:44.420632 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:44.420641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:44.420704 (+ 63us) server_negotiation.cc:407] Connection header received
0925 12:21:44.420877 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:44.420885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:44.420973 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:44.422070 (+ 1097us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:44.422194 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:44.423649 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:44.425074 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:44.425972 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:44.426156 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:44.429453 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:44.429457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:44.429458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:44.429471 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:44.431617 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:44.432055 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:44.432059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:44.432059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:44.432182 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:44.432381 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:44.432382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:44.432382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:44.432689 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:44.432866 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:44.432950 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 12:21:44.433104 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":64,"threads_started":1}
I20260925 12:21:44.434190 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:21:44.445302 4210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b5a068c38b4ec08e6dc6e9c033e9ec: Generated new TSK 0
I20260925 12:21:44.445530 4210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:21:44.453927 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:44.449685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.1.47.190:40273)
0925 12:21:44.449732 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:44.449735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:44.449844 (+ 109us) server_negotiation.cc:407] Connection header received
0925 12:21:44.449917 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:44.449918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:44.449932 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:44.449984 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:44.450343 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:44.451154 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:44.452153 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:44.452492 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:44.452569 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:44.453387 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:44.453388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:44.453389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:44.453635 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:44.453663 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:44.453671 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:44.453733 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:44.453811 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 12:21:44.453840 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:21:44.456882 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4123
2026-09-25T12:21:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1326 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4224](info): set up 2 sockets
Sep 25 12:21:44 dist-test-slave-l9ww krb5kdc[4224](info): commencing operation
krb5kdc: starting...
W20260925 12:21:45.539762 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:45 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:45.668063 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46329
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:45.750082 4237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:45.750365 4237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:45.750403 4237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:45.751657 4237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:45.751754 4237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:45.751778 4237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:45.751799 4237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:45.751825 4237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:45.753438 4237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:45.753750 4237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:45.754000 4237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:45.759127 4243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:45.759127 4245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:45.759137 4242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:45.759138 4237 server_base.cc:1034] running on GCE node
I20260925 12:21:45.759495 4237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:45.759765 4237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:45.760958 4237 hybrid_clock.cc:648] HybridClock initialized: now 1790338905760940 us; error 29 us; skew 500 ppm
Sep 25 12:21:45 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:45.763705 4237 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:45.764215 4237 webserver.cc:460] Webserver started at http://127.1.47.190:44583/ using document root <none> and password file <none>
I20260925 12:21:45.764405 4237 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:45.764444 4237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:45.764533 4237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:45.765584 4237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "ae9370370e094ce59537c1e968b69445"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "01af7076453f878b9e3629ee4fa0afeb"
server_key_iv: "621d6325a1aaada00170679cb0cfdb66"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.765885 4237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "ae9370370e094ce59537c1e968b69445"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "01af7076453f878b9e3629ee4fa0afeb"
server_key_iv: "621d6325a1aaada00170679cb0cfdb66"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.767227 4237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:45.768123 4252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:45.768350 4237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:45.768428 4237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "ae9370370e094ce59537c1e968b69445"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "01af7076453f878b9e3629ee4fa0afeb"
server_key_iv: "621d6325a1aaada00170679cb0cfdb66"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.768483 4237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:45.777300 4237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:45.779451 4237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:45.779582 4237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:45.784309 4237 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46329
I20260925 12:21:45.784406 4304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46329 every 8 connection(s)
I20260925 12:21:45.784744 4237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:45.785863 4305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:45.788615 4305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445: Bootstrap starting.
I20260925 12:21:45.789206 4305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:45.789465 4305 log.cc:824] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:45.790266 4305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445: No bootstrap required, opened a new log
I20260925 12:21:45.792120 4305 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } }
I20260925 12:21:45.792282 4305 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:45.792308 4305 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9370370e094ce59537c1e968b69445, State: Initialized, Role: FOLLOWER
I20260925 12:21:45.792430 4305 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } }
I20260925 12:21:45.792527 4305 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:45.792567 4305 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:45.792622 4305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:45.793197 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4237
I20260925 12:21:45.793330 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:45.793587 1214 external_mini_cluster.cc:1428] Setting key 2b855a5c6f15ada1b41c03c4658a85c1
I20260925 12:21:45.793871 4305 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } }
I20260925 12:21:45.793975 4305 leader_election.cc:302] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae9370370e094ce59537c1e968b69445; no voters:
I20260925 12:21:45.794230 4305 leader_election.cc:288] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:45.794260 4308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:45.794606 4305 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:45.794690 4308 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [term 1 LEADER]: Becoming Leader. State: Replica: ae9370370e094ce59537c1e968b69445, State: Running, Role: LEADER
I20260925 12:21:45.794837 4308 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } }
I20260925 12:21:45.795336 4311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae9370370e094ce59537c1e968b69445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } } }
I20260925 12:21:45.795428 4311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:45.795563 4312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae9370370e094ce59537c1e968b69445. Latest consensus state: current_term: 1 leader_uuid: "ae9370370e094ce59537c1e968b69445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9370370e094ce59537c1e968b69445" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46329 } } }
I20260925 12:21:45.795651 4312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:45.795930 4319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:45.796679 4319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:45.798523 4319 catalog_manager.cc:1347] Generated new cluster ID: 371805e0c39f4237b871325703ded624
I20260925 12:21:45.798640 4319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:45 dist-test-slave-l9ww krb5kdc[4224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:45.806849 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:45.794163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45762 (local address 127.1.47.190:46329)
0925 12:21:45.794336 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:45.794347 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:45.794573 (+ 226us) server_negotiation.cc:407] Connection header received
0925 12:21:45.794724 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:45.794732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:45.794787 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:45.794920 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:45.795911 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:45.797181 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:45.798254 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:45.798968 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:45.799102 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:45.801927 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:45.801932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:45.801933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:45.801946 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:45.804888 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:45.805321 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:45.805325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:45.805325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:45.805480 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:45.805707 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:45.805708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:45.805708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:45.806101 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:45.806296 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:45.806398 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 12:21:45.806575 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260925 12:21:45.829603 4319 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:45.830461 4319 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:45.841475 4319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae9370370e094ce59537c1e968b69445: Generated new TSK 0
I20260925 12:21:45.841724 4319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:45.876205 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:45.961009 4331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:45.961272 4331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:45.961310 4331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:45.962478 4331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:45.962558 4331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:45.962601 4331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:45.964114 4331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:45.964406 4331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:45.964658 4331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:45.969911 4339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:45.969962 4337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:45.969924 4336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:45.970148 4331 server_base.cc:1034] running on GCE node
I20260925 12:21:45.970468 4331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:45.970727 4331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:45.972005 4331 hybrid_clock.cc:648] HybridClock initialized: now 1790338905971983 us; error 32 us; skew 500 ppm
Sep 25 12:21:45 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:45.979624 4331 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:45.980234 4331 webserver.cc:460] Webserver started at http://127.1.47.129:41917/ using document root <none> and password file <none>
I20260925 12:21:45.980439 4331 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:45.980490 4331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:45.980593 4331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:45.981729 4331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "06b1045ca9b942d48f7d39ff86e7a5b7"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "69840385e632476fd0569e1ec72cec37"
server_key_iv: "bd073abd3f53dd6cff5e694111e70084"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.982057 4331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "06b1045ca9b942d48f7d39ff86e7a5b7"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "69840385e632476fd0569e1ec72cec37"
server_key_iv: "bd073abd3f53dd6cff5e694111e70084"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.983440 4331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:45.984309 4346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:45.984526 4331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 12:21:45.984642 4331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "06b1045ca9b942d48f7d39ff86e7a5b7"
format_stamp: "Formatted at 2026-09-25 12:21:45 on dist-test-slave-l9ww"
server_key: "69840385e632476fd0569e1ec72cec37"
server_key_iv: "bd073abd3f53dd6cff5e694111e70084"
server_key_version: "encryptionkey@0"
I20260925 12:21:45.984727 4331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:46.006255 4331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:46.008831 4331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:46.009034 4331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:46.009291 4331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:46.009639 4331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:46.009699 4331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.009735 4331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:46.009763 4331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.015723 4331 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:36717
I20260925 12:21:46.015838 4459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:36717 every 8 connection(s)
I20260925 12:21:46.016199 4331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:46.022228 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4331
I20260925 12:21:46.022341 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:46.022583 1214 external_mini_cluster.cc:1428] Setting key 43ae29afcc186d45fa7cb434ed06c61d
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:46.027042 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.017107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:57161 (local address 127.1.47.190:46329)
0925 12:21:46.017166 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:46.017168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:46.017411 (+ 243us) server_negotiation.cc:407] Connection header received
0925 12:21:46.017936 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:46.017937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:46.017946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:46.018012 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:46.019027 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.019533 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.020732 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.020937 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.021004 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.023684 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:46.023685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:46.023686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:46.023694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:46.025482 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.025969 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.025970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.025970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.026023 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.026326 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.026327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.026327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.026450 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:46.026533 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:46.026826 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 12:21:46.026934 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:21:46.027218 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.017070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46329 (local address 127.1.47.129:57161)
0925 12:21:46.017255 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:46.017302 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:46.017843 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:46.018071 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:46.018075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:46.018703 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:46.018916 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.018921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.019638 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.019639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.020640 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.020643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.021021 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.021022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.021130 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.021747 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:46.021756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:46.023578 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:46.025574 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.025580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.025582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.025886 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.026117 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.026119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.026119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.026244 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.026523 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:46.026530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:46.026685 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 12:21:46.026803 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 12:21:46.026970 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260925 12:21:46.027720 4460 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46329
I20260925 12:21:46.027835 4460 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:46.028097 4460 heartbeater.cc:511] Master 127.1.47.190:46329 requested a full tablet report, sending...
I20260925 12:21:46.028663 4269 ts_manager.cc:194] Registered new tserver with Master: 06b1045ca9b942d48f7d39ff86e7a5b7 (127.1.47.129:36717)
I20260925 12:21:46.029783 4269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:57161
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:46.051342 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:46.146927 4465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:46.147153 4465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:46.147178 4465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:46.148444 4465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:46.148496 4465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:46.148531 4465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:46.150086 4465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:46.150369 4465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:46.150631 4465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:46.156291 4473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:46.156271 4471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:46.156255 4470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:46.156527 4465 server_base.cc:1034] running on GCE node
I20260925 12:21:46.156769 4465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:46.157030 4465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:46.158267 4465 hybrid_clock.cc:648] HybridClock initialized: now 1790338906158254 us; error 31 us; skew 500 ppm
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:46.161296 4465 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:46.161810 4465 webserver.cc:460] Webserver started at http://127.1.47.130:44033/ using document root <none> and password file <none>
I20260925 12:21:46.162081 4465 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:46.162163 4465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:46.162328 4465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:46.163503 4465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "6360d9f5cb8b4a478098de58e8a5f02a"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "af4dcf29a0f3e943ee7c52c321a8ede6"
server_key_iv: "1d7386f2141522dcbcd86f1032f70336"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.163995 4465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "6360d9f5cb8b4a478098de58e8a5f02a"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "af4dcf29a0f3e943ee7c52c321a8ede6"
server_key_iv: "1d7386f2141522dcbcd86f1032f70336"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.165583 4465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:46.166729 4480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.166936 4465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:46.167048 4465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "6360d9f5cb8b4a478098de58e8a5f02a"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "af4dcf29a0f3e943ee7c52c321a8ede6"
server_key_iv: "1d7386f2141522dcbcd86f1032f70336"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.167136 4465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:46.186146 4465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:46.188927 4465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:46.189096 4465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:46.189363 4465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:46.189739 4465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:46.189810 4465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.189851 4465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:46.189881 4465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.196952 4465 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:33553
I20260925 12:21:46.196991 4593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:33553 every 8 connection(s)
I20260925 12:21:46.197374 4465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:46.207239 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4465
I20260925 12:21:46.207370 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:46.207572 1214 external_mini_cluster.cc:1428] Setting key 8567e5038ad9c369c45678e90b82c7cc
I20260925 12:21:46.209223 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.198277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:41079 (local address 127.1.47.190:46329)
0925 12:21:46.198390 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:46.198391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:46.198633 (+ 242us) server_negotiation.cc:407] Connection header received
0925 12:21:46.199175 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:46.199176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:46.199188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:46.199266 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:46.200451 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.201030 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.202243 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.202473 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.202542 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.205413 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:46.205415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:46.205416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:46.205424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:46.207147 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.207751 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.207753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.207753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.207834 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.208195 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.208197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.208197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.208351 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:46.208430 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:46.208893 (+ 463us) server_negotiation.cc:299] Negotiation successful
0925 12:21:46.209059 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 12:21:46.209934 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.198254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46329 (local address 127.1.47.130:41079)
0925 12:21:46.198448 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:46.198496 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:46.199048 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:46.199328 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:46.199332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:46.200083 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:46.200305 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.200311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.201167 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.201168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.202133 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.202135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.202581 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.202583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.202715 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.203365 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:46.203374 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:46.205265 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:46.207254 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.207262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.207265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.207655 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.207968 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.207969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.207969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.208117 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.208456 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:46.208465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:46.208693 (+ 228us) client_negotiation.cc:769] Sending connection context
0925 12:21:46.209366 (+ 673us) client_negotiation.cc:240] Negotiation successful
0925 12:21:46.209596 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260925 12:21:46.210505 4594 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46329
I20260925 12:21:46.210610 4594 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:46.210839 4594 heartbeater.cc:511] Master 127.1.47.190:46329 requested a full tablet report, sending...
I20260925 12:21:46.211252 4269 ts_manager.cc:194] Registered new tserver with Master: 6360d9f5cb8b4a478098de58e8a5f02a (127.1.47.130:33553)
I20260925 12:21:46.211684 4269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:41079
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:46.235744 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:46.316982 4599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:46.317250 4599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:46.317292 4599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:46.318485 4599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:46.318558 4599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:46.318606 4599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:46.320129 4599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:46.320398 4599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:46.320638 4599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:46.325847 4604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:46.325837 4605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:46.325835 4607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:46.326081 4599 server_base.cc:1034] running on GCE node
I20260925 12:21:46.326356 4599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:46.326647 4599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:46.327808 4599 hybrid_clock.cc:648] HybridClock initialized: now 1790338906327787 us; error 42 us; skew 500 ppm
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:46.336805 4599 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:46.337342 4599 webserver.cc:460] Webserver started at http://127.1.47.131:41607/ using document root <none> and password file <none>
I20260925 12:21:46.337627 4599 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:46.337680 4599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:46.337870 4599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:46.339052 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "d9a2f300e2d84fa3b0f1fdd3f803ed6f"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "31bb51c84ee86d53a289ea723e1a36f1"
server_key_iv: "79cb9a17c7208cb56fd4b5ca9126ce1e"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.339480 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "d9a2f300e2d84fa3b0f1fdd3f803ed6f"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "31bb51c84ee86d53a289ea723e1a36f1"
server_key_iv: "79cb9a17c7208cb56fd4b5ca9126ce1e"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.341132 4599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260925 12:21:46.342092 4614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.342303 4599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:46.342387 4599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "d9a2f300e2d84fa3b0f1fdd3f803ed6f"
format_stamp: "Formatted at 2026-09-25 12:21:46 on dist-test-slave-l9ww"
server_key: "31bb51c84ee86d53a289ea723e1a36f1"
server_key_iv: "79cb9a17c7208cb56fd4b5ca9126ce1e"
server_key_version: "encryptionkey@0"
I20260925 12:21:46.342492 4599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:46.366701 4599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:46.369261 4599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:46.369480 4599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:46.369702 4599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:46.369966 4599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:46.369997 4599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.370020 4599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:46.370033 4599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:46.376003 4599 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:40547
I20260925 12:21:46.376065 4727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:40547 every 8 connection(s)
I20260925 12:21:46.376396 4599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:46.381217 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4599
I20260925 12:21:46.381315 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:46.381500 1214 external_mini_cluster.cc:1428] Setting key 1b917be264c2477988a3c05814301cdb
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:46.386200 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.377337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:51363 (local address 127.1.47.190:46329)
0925 12:21:46.377396 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:46.377398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:46.377600 (+ 202us) server_negotiation.cc:407] Connection header received
0925 12:21:46.378068 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:46.378070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:46.378079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:46.378146 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:46.379216 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.379755 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.380877 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.381089 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.381155 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.383689 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:46.383691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:46.383691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:46.383698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:46.384959 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.385330 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.385330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.385331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.385381 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.385605 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.385605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.385606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.385711 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:46.385762 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:46.386063 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 12:21:46.386135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:21:46.386272 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.377284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46329 (local address 127.1.47.131:51363)
0925 12:21:46.377456 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:46.377500 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:46.377971 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:46.378239 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:46.378243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:46.378837 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:46.379057 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.379063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.379912 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.379913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.380786 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:46.380788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.381203 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.381204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:46.381368 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.381979 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:46.381991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:46.383601 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:46.385036 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.385041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.385043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.385262 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.385440 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:46.385440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:46.385441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:46.385548 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:46.385767 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:46.385771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:46.385889 (+ 118us) client_negotiation.cc:769] Sending connection context
0925 12:21:46.385973 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:21:46.386102 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260925 12:21:46.386732 4728 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46329
I20260925 12:21:46.386817 4728 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:46.387032 4728 heartbeater.cc:511] Master 127.1.47.190:46329 requested a full tablet report, sending...
I20260925 12:21:46.387441 4269 ts_manager.cc:194] Registered new tserver with Master: d9a2f300e2d84fa3b0f1fdd3f803ed6f (127.1.47.131:40547)
I20260925 12:21:46.387820 4269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:51363
I20260925 12:21:46.394722 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338906, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:46.418803 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:46.411176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.1.47.190:46329)
0925 12:21:46.411248 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:46.411251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:46.411360 (+ 109us) server_negotiation.cc:407] Connection header received
0925 12:21:46.411409 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:46.411410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:46.411421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:46.411468 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:46.412084 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.412619 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.413334 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:46.413536 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:46.413596 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:46.416034 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:46.416035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:46.416036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:46.416042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:46.417361 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.417702 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.417703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.417704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.417771 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:46.418019 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:46.418020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:46.418021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:46.418265 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:46.418393 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:46.418554 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 12:21:46.418655 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 12:21:46.419035 4269 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45766
I20260925 12:21:46.419552 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4331
I20260925 12:21:46.427452 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4465
I20260925 12:21:46.432822 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4599
I20260925 12:21:46.437350 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4237
2026-09-25T12:21:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1982 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4740](info): set up 2 sockets
Sep 25 12:21:46 dist-test-slave-l9ww krb5kdc[4740](info): commencing operation
krb5kdc: starting...
W20260925 12:21:47.507827 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:47 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:47.636240 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45715
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:45715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:47.717654 4753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:47.717974 4753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:47.718019 4753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:47.719326 4753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:47.719408 4753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:47.719430 4753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:47.719450 4753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:47.719482 4753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:47.721079 4753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:45715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:47.721380 4753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:47.721618 4753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:47.726559 4759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:47.726555 4761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:47.726557 4758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:47.726887 4753 server_base.cc:1034] running on GCE node
I20260925 12:21:47.727124 4753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:47.727423 4753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:47.728583 4753 hybrid_clock.cc:648] HybridClock initialized: now 1790338907728573 us; error 34 us; skew 500 ppm
Sep 25 12:21:47 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338907, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:47.731210 4753 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:47.731727 4753 webserver.cc:460] Webserver started at http://127.1.47.190:33389/ using document root <none> and password file <none>
I20260925 12:21:47.731994 4753 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:47.732043 4753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:47.732190 4753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:47.733297 4753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "cbceb0a88d5b612c3ba073288f849d2d"
server_key_iv: "4fdbc43ed9a31d1a6091247c16862755"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.733683 4753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "cbceb0a88d5b612c3ba073288f849d2d"
server_key_iv: "4fdbc43ed9a31d1a6091247c16862755"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.735100 4753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:47.736030 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:47.736234 4753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:47.736373 4753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "cbceb0a88d5b612c3ba073288f849d2d"
server_key_iv: "4fdbc43ed9a31d1a6091247c16862755"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.736485 4753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:47.756685 4753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:47.759053 4753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:47.759255 4753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:47.763284 4753 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:45715
I20260925 12:21:47.763373 4820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:45715 every 8 connection(s)
I20260925 12:21:47.763906 4753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:47.764729 4821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:47.767391 4821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff: Bootstrap starting.
I20260925 12:21:47.768182 4821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:47.768549 4821 log.cc:824] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:47.769383 4821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff: No bootstrap required, opened a new log
I20260925 12:21:47.771034 4821 raft_consensus.cc:348] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } }
I20260925 12:21:47.771178 4821 raft_consensus.cc:374] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:47.771200 4821 raft_consensus.cc:729] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda22d2a53974aa4a0e6d8b13b8e07ff, State: Initialized, Role: FOLLOWER
I20260925 12:21:47.771248 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4753
I20260925 12:21:47.771382 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:47.771404 4821 consensus_queue.cc:260] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } }
I20260925 12:21:47.771519 4821 raft_consensus.cc:388] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:47.771581 4821 raft_consensus.cc:482] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:47.771631 1214 external_mini_cluster.cc:1428] Setting key e1e49a82a7714b06118a5902a5aeb707
I20260925 12:21:47.771633 4821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:47.772403 4821 raft_consensus.cc:504] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } }
I20260925 12:21:47.772542 4821 leader_election.cc:302] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bda22d2a53974aa4a0e6d8b13b8e07ff; no voters:
I20260925 12:21:47.772714 4821 leader_election.cc:288] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:47.772745 4826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:47.772894 4826 raft_consensus.cc:686] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [term 1 LEADER]: Becoming Leader. State: Replica: bda22d2a53974aa4a0e6d8b13b8e07ff, State: Running, Role: LEADER
I20260925 12:21:47.772969 4821 sys_catalog.cc:564] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:47.773011 4826 consensus_queue.cc:237] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } }
I20260925 12:21:47.773582 4827 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader bda22d2a53974aa4a0e6d8b13b8e07ff. Latest consensus state: current_term: 1 leader_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } } }
I20260925 12:21:47.773672 4827 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:47.773574 4828 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda22d2a53974aa4a0e6d8b13b8e07ff" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45715 } } }
I20260925 12:21:47.773823 4828 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:47.773991 4835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:47.774729 4835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:47.776998 4835 catalog_manager.cc:1347] Generated new cluster ID: f58e390681d3481ba65ac040b2f917bb
I20260925 12:21:47.777055 4835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:47 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:47.783432 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:47.772578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.1.47.190:45715)
0925 12:21:47.772775 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:47.772787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:47.772827 (+ 40us) server_negotiation.cc:407] Connection header received
0925 12:21:47.772958 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:47.772966 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:47.773026 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:47.774039 (+ 1013us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:47.774193 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.775537 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.776625 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.777167 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.777305 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:47.779663 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:47.779667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:47.779668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:47.779685 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:47.781456 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:47.781858 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:47.781861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:47.781862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:47.782156 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:47.782430 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:47.782431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:47.782431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:47.782715 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:47.782979 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:47.783055 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 12:21:47.783218 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260925 12:21:47.822432 4835 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:47.823191 4835 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:47.825423 4835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bda22d2a53974aa4a0e6d8b13b8e07ff: Generated new TSK 0
I20260925 12:21:47.825651 4835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:47.864741 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:47.946919 4847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:47.947178 4847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:47.947219 4847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:47.948436 4847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:47.948510 4847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:47.948560 4847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:47.950119 4847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:47.950394 4847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:47.950635 4847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:47.955694 4855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:47.955726 4852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:47.955936 4847 server_base.cc:1034] running on GCE node
W20260925 12:21:47.955731 4853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:47.956236 4847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:47.956518 4847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:47.957672 4847 hybrid_clock.cc:648] HybridClock initialized: now 1790338907957661 us; error 46 us; skew 500 ppm
Sep 25 12:21:47 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338907, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:47.960441 4847 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:47.960961 4847 webserver.cc:460] Webserver started at http://127.1.47.129:37525/ using document root <none> and password file <none>
I20260925 12:21:47.961207 4847 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:47.961253 4847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:47.961427 4847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:47.962476 4847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "86111cb3354e4e738f84139beae23010"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "b5302223047e10cb1ca093cda3b52b04"
server_key_iv: "e5e9ac32e42cde01391c218c89670f90"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.962831 4847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "86111cb3354e4e738f84139beae23010"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "b5302223047e10cb1ca093cda3b52b04"
server_key_iv: "e5e9ac32e42cde01391c218c89670f90"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.964184 4847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:47.964946 4862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:47.965116 4847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:47.965207 4847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "86111cb3354e4e738f84139beae23010"
format_stamp: "Formatted at 2026-09-25 12:21:47 on dist-test-slave-l9ww"
server_key: "b5302223047e10cb1ca093cda3b52b04"
server_key_iv: "e5e9ac32e42cde01391c218c89670f90"
server_key_version: "encryptionkey@0"
I20260925 12:21:47.965289 4847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:47.983042 4847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:47.985675 4847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:47.985862 4847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:47.986120 4847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:47.986459 4847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:47.986516 4847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:47.986559 4847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:47.986586 4847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:47.992734 4847 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:34739
I20260925 12:21:47.992782 4975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:34739 every 8 connection(s)
I20260925 12:21:47.993115 4847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:21:47 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338907, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:48.000085 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4847
I20260925 12:21:48.000195 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:48.000437 1214 external_mini_cluster.cc:1428] Setting key 9f1a08092e543ae1368ab9e7899f012e
I20260925 12:21:48.003363 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:47.993846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:39121 (local address 127.1.47.190:45715)
0925 12:21:47.993890 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:47.993892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:47.994189 (+ 297us) server_negotiation.cc:407] Connection header received
0925 12:21:47.995184 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:47.995185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:47.995201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:47.995256 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:47.995974 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.996491 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.997536 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.997743 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.997859 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.000244 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:48.000246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:48.000246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:48.000255 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:48.001903 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.002341 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.002343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.002343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.002400 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.002662 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.002664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.002664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.002799 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:48.002874 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:48.003176 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 12:21:48.003262 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:21:48.003554 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:47.993844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:45715 (local address 127.1.47.129:39121)
0925 12:21:47.994050 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:47.994088 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:47.995052 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:47.995333 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:47.995337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:47.995612 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:47.995818 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:47.995822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.996592 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.996593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:47.997426 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:47.997428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:47.997865 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:47.997866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:47.998000 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:47.998562 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:47.998571 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:48.000102 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:48.002010 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.002018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.002020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.002258 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.002473 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.002474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.002475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.002593 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.002879 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:48.002886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:48.003033 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 12:21:48.003143 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 12:21:48.003311 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
I20260925 12:21:48.004025 4976 heartbeater.cc:347] Connected to a master server at 127.1.47.190:45715
I20260925 12:21:48.004139 4976 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:48.004345 4976 heartbeater.cc:511] Master 127.1.47.190:45715 requested a full tablet report, sending...
I20260925 12:21:48.004904 4785 ts_manager.cc:194] Registered new tserver with Master: 86111cb3354e4e738f84139beae23010 (127.1.47.129:34739)
I20260925 12:21:48.005507 4785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:39121
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:48.028041 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:48.111333 4981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:48.111577 4981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:48.111603 4981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:48.112921 4981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:48.112974 4981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:48.113001 4981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:48.114564 4981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:48.114856 4981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:48.115113 4981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:48.120287 4987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:48.120329 4986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:48.120286 4989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:48.120558 4981 server_base.cc:1034] running on GCE node
I20260925 12:21:48.120781 4981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:48.121053 4981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:48.122246 4981 hybrid_clock.cc:648] HybridClock initialized: now 1790338908122222 us; error 37 us; skew 500 ppm
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:48.125226 4981 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:48.125604 4981 webserver.cc:460] Webserver started at http://127.1.47.130:40583/ using document root <none> and password file <none>
I20260925 12:21:48.125845 4981 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:48.125892 4981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:48.126024 4981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:48.127086 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "14349a0a50c6446ca84c50a694b8b3eb"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "1f1cbd17c3340b88671bee8bc0c848d3"
server_key_iv: "9e2cd4f7bf16c0ed57f449f850203bcb"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.127445 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "14349a0a50c6446ca84c50a694b8b3eb"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "1f1cbd17c3340b88671bee8bc0c848d3"
server_key_iv: "9e2cd4f7bf16c0ed57f449f850203bcb"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.128899 4981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:48.129652 4996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.129849 4981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:48.129940 4981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "14349a0a50c6446ca84c50a694b8b3eb"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "1f1cbd17c3340b88671bee8bc0c848d3"
server_key_iv: "9e2cd4f7bf16c0ed57f449f850203bcb"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.130009 4981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:48.155375 4981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:48.157793 4981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:48.157936 4981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:48.158171 4981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:48.158483 4981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:48.158545 4981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.158588 4981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:48.158613 4981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.164422 4981 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:35437
I20260925 12:21:48.164484 5109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:35437 every 8 connection(s)
I20260925 12:21:48.164840 4981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:48.173282 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 4981
I20260925 12:21:48.173408 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:48.173688 1214 external_mini_cluster.cc:1428] Setting key 3536973de91e21a24d31c4a1eae262f9
I20260925 12:21:48.175524 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.165612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:45715 (local address 127.1.47.130:40679)
0925 12:21:48.165776 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:48.165817 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:48.166700 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:48.166924 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:48.166927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:48.167176 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:48.167391 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:48.167397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.168220 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.168221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:48.169136 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:48.169138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.169648 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.169649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:48.169770 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.170348 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:48.170356 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:48.171769 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:48.173879 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.173888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.173890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.174246 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.174499 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.174506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.174506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.174663 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.174992 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:48.174997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:48.175127 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 12:21:48.175217 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 12:21:48.175352 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260925 12:21:48.175544 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.165667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:40679 (local address 127.1.47.190:45715)
0925 12:21:48.165744 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:48.165746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:48.165918 (+ 172us) server_negotiation.cc:407] Connection header received
0925 12:21:48.166791 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:48.166792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:48.166801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:48.166866 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:48.167522 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.168113 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.169259 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.169540 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.169632 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.171932 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:48.171934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:48.171934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:48.171943 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:48.173784 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.174337 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.174338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.174339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.174421 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.174743 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.174744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.174745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.174919 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:48.174999 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:48.175268 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 12:21:48.175386 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 12:21:48.176752 5110 heartbeater.cc:347] Connected to a master server at 127.1.47.190:45715
I20260925 12:21:48.176862 5110 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:48.177093 5110 heartbeater.cc:511] Master 127.1.47.190:45715 requested a full tablet report, sending...
I20260925 12:21:48.177573 4785 ts_manager.cc:194] Registered new tserver with Master: 14349a0a50c6446ca84c50a694b8b3eb (127.1.47.130:35437)
I20260925 12:21:48.177999 4785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:40679
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:48.203946 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:48.285744 5115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:48.286001 5115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:48.286041 5115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:48.287178 5115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:48.287251 5115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:48.287297 5115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:48.288858 5115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:45715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:48.289124 5115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:48.289353 5115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 12:21:48.294500 5115 server_base.cc:1034] running on GCE node
W20260925 12:21:48.294514 5121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:48.294632 5123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:48.294509 5120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:48.294989 5115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:48.295239 5115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:48.296404 5115 hybrid_clock.cc:648] HybridClock initialized: now 1790338908296397 us; error 50 us; skew 500 ppm
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:48.298897 5115 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:48.299311 5115 webserver.cc:460] Webserver started at http://127.1.47.131:33051/ using document root <none> and password file <none>
I20260925 12:21:48.299537 5115 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:48.299607 5115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:48.299757 5115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:48.300839 5115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "056cc3575aaa4bf68292a28766d8fe52"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "59221e375440ed5ef22d11c0063a1a01"
server_key_iv: "0e70347a4d9b42e1df01b1be621d2fe1"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.301206 5115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "056cc3575aaa4bf68292a28766d8fe52"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "59221e375440ed5ef22d11c0063a1a01"
server_key_iv: "0e70347a4d9b42e1df01b1be621d2fe1"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.302515 5115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:21:48.303263 5130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.303457 5115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:48.303553 5115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "056cc3575aaa4bf68292a28766d8fe52"
format_stamp: "Formatted at 2026-09-25 12:21:48 on dist-test-slave-l9ww"
server_key: "59221e375440ed5ef22d11c0063a1a01"
server_key_iv: "0e70347a4d9b42e1df01b1be621d2fe1"
server_key_version: "encryptionkey@0"
I20260925 12:21:48.303634 5115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:48.326936 5115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:48.329243 5115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:48.329418 5115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:48.329650 5115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:48.329975 5115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:48.330031 5115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.330072 5115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:48.330101 5115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:48.336068 5115 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:40679
I20260925 12:21:48.336109 5243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:40679 every 8 connection(s)
I20260925 12:21:48.336455 5115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:48.338822 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5115
I20260925 12:21:48.338893 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:48.339076 1214 external_mini_cluster.cc:1428] Setting key 7308341d7e6ac774d8073bea2c10302b
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:48.346263 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.337260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:55373 (local address 127.1.47.190:45715)
0925 12:21:48.337349 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:48.337351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:48.337523 (+ 172us) server_negotiation.cc:407] Connection header received
0925 12:21:48.338424 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:48.338425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:48.338435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:48.338489 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:48.339199 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.339929 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.340967 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.341179 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.341267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.343417 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:48.343419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:48.343420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:48.343428 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:48.344862 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.345298 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.345299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.345299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.345360 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.345617 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.345618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.345618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.345740 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:48.345818 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:48.346091 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 12:21:48.346169 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 12:21:48.346477 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.337214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:45715 (local address 127.1.47.131:55373)
0925 12:21:48.337406 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:48.337442 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:48.338328 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:48.338543 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:48.338547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:48.338819 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:48.339087 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:48.339092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.340027 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.340028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:48.340869 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:48.340871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.341276 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.341277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:48.341418 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.341907 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:48.341915 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:48.343308 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:48.344970 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.344976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.344979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.345206 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.345436 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:48.345437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:48.345437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:48.345548 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:48.345808 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:48.345814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:48.345957 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 12:21:48.346070 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 12:21:48.346229 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260925 12:21:48.346997 5244 heartbeater.cc:347] Connected to a master server at 127.1.47.190:45715
I20260925 12:21:48.347083 5244 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:48.347295 5244 heartbeater.cc:511] Master 127.1.47.190:45715 requested a full tablet report, sending...
I20260925 12:21:48.347745 4785 ts_manager.cc:194] Registered new tserver with Master: 056cc3575aaa4bf68292a28766d8fe52 (127.1.47.131:40679)
I20260925 12:21:48.348156 4785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:55373
I20260925 12:21:48.350970 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:48.375506 5251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.366937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.1.47.129:34739)
0925 12:21:48.367107 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:48.367112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:48.367122 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:48.367149 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:48.367151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:48.367165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:48.367264 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:48.367954 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.368711 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.369574 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.369918 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.370024 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.372462 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:48.372472 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:48.372472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:48.372481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:48.374247 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.374603 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.374607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.374607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.374708 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.374924 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.374925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.374925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.375082 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:48.375251 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:48.375298 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 12:21:48.375380 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[4740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:48.383399 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:48.376848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.1.47.190:45715)
0925 12:21:48.376890 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:48.376893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:48.376934 (+ 41us) server_negotiation.cc:407] Connection header received
0925 12:21:48.376990 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:48.376990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:48.377000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:48.377050 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:48.377528 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.378022 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.378662 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:48.378841 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:48.378896 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:48.381155 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:48.381156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:48.381156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:48.381162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:48.382382 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.382681 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.382682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.382682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.382729 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:48.382897 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:48.382897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:48.382897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:48.383035 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:48.383108 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:48.383265 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 12:21:48.383326 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 12:21:48.383661 4785 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170
I20260925 12:21:48.384070 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4847
I20260925 12:21:48.388602 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4981
I20260925 12:21:48.393530 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5115
I20260925 12:21:48.397925 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 4753
2026-09-25T12:21:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1959 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[5256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[5256](info): set up 2 sockets
Sep 25 12:21:48 dist-test-slave-l9ww krb5kdc[5256](info): commencing operation
krb5kdc: starting...
W20260925 12:21:49.515820 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:49 dist-test-slave-l9ww krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:49.643846 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40529
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:40529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:49.725025 5269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:49.725297 5269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:49.725340 5269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:49.726555 5269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:49.726649 5269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:49.726675 5269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:49.726697 5269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:49.726727 5269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:49.728372 5269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:40529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:49.728686 5269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:49.728924 5269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:49.734000 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:49.734000 5277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:49.734187 5269 server_base.cc:1034] running on GCE node
W20260925 12:21:49.734036 5274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:49.734547 5269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:49.734889 5269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:49.736083 5269 hybrid_clock.cc:648] HybridClock initialized: now 1790338909736053 us; error 42 us; skew 500 ppm
Sep 25 12:21:49 dist-test-slave-l9ww krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338909, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:49.738749 5269 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:49.739133 5269 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:49.739216 5269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:49.739365 5269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:49.740535 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "b4f9847afeb14bba9751726747308b86"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "334fdd5a2f94d3b732872868832adf89"
server_key_iv: "4d81f009bab007d95e5a7d2b174d842c"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.740906 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "b4f9847afeb14bba9751726747308b86"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "334fdd5a2f94d3b732872868832adf89"
server_key_iv: "4d81f009bab007d95e5a7d2b174d842c"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.742265 5269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:49.743040 5283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:49.743222 5269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:49.743328 5269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "b4f9847afeb14bba9751726747308b86"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "334fdd5a2f94d3b732872868832adf89"
server_key_iv: "4d81f009bab007d95e5a7d2b174d842c"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.743414 5269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:49.768395 5269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:49.770794 5269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:49.770974 5269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:49.774901 5269 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:40529
I20260925 12:21:49.774943 5335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:40529 every 8 connection(s)
I20260925 12:21:49.775254 5269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:49.776300 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:49.778618 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: Bootstrap starting.
I20260925 12:21:49.779239 5336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:49.779453 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5269
I20260925 12:21:49.779569 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:49.779577 5336 log.cc:824] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:49.779759 1214 external_mini_cluster.cc:1428] Setting key 1965f77005bef99d18ad0242a900f5a3
I20260925 12:21:49.780429 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: No bootstrap required, opened a new log
I20260925 12:21:49.783196 5336 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } }
I20260925 12:21:49.783408 5336 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:49.783445 5336 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f9847afeb14bba9751726747308b86, State: Initialized, Role: FOLLOWER
I20260925 12:21:49.783576 5336 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } }
I20260925 12:21:49.783648 5336 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:49.783677 5336 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:49.783720 5336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:49.784476 5336 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } }
I20260925 12:21:49.784575 5336 leader_election.cc:302] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4f9847afeb14bba9751726747308b86; no voters:
I20260925 12:21:49.784796 5336 leader_election.cc:288] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:49.784825 5341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:49.785003 5341 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [term 1 LEADER]: Becoming Leader. State: Replica: b4f9847afeb14bba9751726747308b86, State: Running, Role: LEADER
I20260925 12:21:49.785076 5336 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:49.785096 5341 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } }
I20260925 12:21:49.785521 5342 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4f9847afeb14bba9751726747308b86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } } }
I20260925 12:21:49.785590 5342 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:49.785495 5343 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4f9847afeb14bba9751726747308b86. Latest consensus state: current_term: 1 leader_uuid: "b4f9847afeb14bba9751726747308b86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f9847afeb14bba9751726747308b86" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40529 } } }
I20260925 12:21:49.785746 5343 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:49.785914 5347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:49.786418 5347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 12:21:49 dist-test-slave-l9ww krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:49.788554 5347 catalog_manager.cc:1347] Generated new cluster ID: cab15513c94748a7b327a67c29e8272a
I20260925 12:21:49.788617 5347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:49.791699 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:49.780302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.1.47.190:40529)
0925 12:21:49.780471 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:49.780498 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:49.780534 (+ 36us) server_negotiation.cc:407] Connection header received
0925 12:21:49.780994 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:49.781001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:49.781052 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:49.781169 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:49.782044 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.782938 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.784066 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.784508 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.784673 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:49.787787 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:49.787791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:49.787791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:49.787804 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:49.789960 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:49.790405 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:49.790409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:49.790409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:49.790527 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:49.790735 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:49.790736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:49.790736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:49.791066 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:49.791235 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:49.791321 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 12:21:49.791469 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260925 12:21:49.800771 5347 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:49.801308 5347 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:49.806028 5347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: Generated new TSK 0
I20260925 12:21:49.806234 5347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:49.838653 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:49.921722 5362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:49.921941 5362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:49.921967 5362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:49.923287 5362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:49.923329 5362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:49.923357 5362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:49.924883 5362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:49.925076 5362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:49.925268 5362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:49.930439 5370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:49.930557 5367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:49.930481 5368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:49.930683 5362 server_base.cc:1034] running on GCE node
I20260925 12:21:49.930935 5362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:49.931231 5362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:49.932413 5362 hybrid_clock.cc:648] HybridClock initialized: now 1790338909932386 us; error 42 us; skew 500 ppm
Sep 25 12:21:49 dist-test-slave-l9ww krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338909, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:49.935293 5362 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:49.935665 5362 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:49.935755 5362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:49.935947 5362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:49.937081 5362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "53bdec81de0a4a65b2ce456c316f10a5"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "5364e204e28e81c36989d426f5de9eed"
server_key_iv: "094c9d2027355f5888175a1a0bfe648a"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.937471 5362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "53bdec81de0a4a65b2ce456c316f10a5"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "5364e204e28e81c36989d426f5de9eed"
server_key_iv: "094c9d2027355f5888175a1a0bfe648a"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.938868 5362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:49.939683 5376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:49.939893 5362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:49.940001 5362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "53bdec81de0a4a65b2ce456c316f10a5"
format_stamp: "Formatted at 2026-09-25 12:21:49 on dist-test-slave-l9ww"
server_key: "5364e204e28e81c36989d426f5de9eed"
server_key_iv: "094c9d2027355f5888175a1a0bfe648a"
server_key_version: "encryptionkey@0"
I20260925 12:21:49.940089 5362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:49.955600 5362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:49.957801 5362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:49.957983 5362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:49.958243 5362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:49.958580 5362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:49.958638 5362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:49.958680 5362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:49.958702 5362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:49.964320 5362 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43777
I20260925 12:21:49.964397 5489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43777 every 8 connection(s)
I20260925 12:21:49.964684 5362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:21:49 dist-test-slave-l9ww krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338909, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:49.973956 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5362
I20260925 12:21:49.974108 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:49.974334 1214 external_mini_cluster.cc:1428] Setting key 794ec82ec8a4abe943a3fe0cdff4b4c7
I20260925 12:21:49.974767 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:49.965472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:46061 (local address 127.1.47.190:40529)
0925 12:21:49.965575 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:49.965578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:49.965773 (+ 195us) server_negotiation.cc:407] Connection header received
0925 12:21:49.966316 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:49.966317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:49.966330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:49.966379 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:49.967399 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.968029 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.969096 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.969323 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.969402 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:49.971773 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:49.971775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:49.971775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:49.971783 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:49.973207 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:49.973646 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:49.973647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:49.973648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:49.973715 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:49.973994 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:49.973995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:49.973995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:49.974162 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:49.974228 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:49.974574 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 12:21:49.974669 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 12:21:49.975004 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:49.965444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40529 (local address 127.1.47.129:46061)
0925 12:21:49.965636 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:49.965673 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:49.966198 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:49.966472 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:49.966475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:49.966762 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:49.967279 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:49.967284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.968134 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.968135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:49.968980 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:49.968983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:49.969421 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:49.969423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:49.969550 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:49.970150 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:49.970158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:49.971649 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:49.973311 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:49.973318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:49.973320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:49.973548 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:49.973809 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:49.973810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:49.973810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:49.973922 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:49.974256 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:49.974267 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:49.974423 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 12:21:49.974555 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 12:21:49.974756 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260925 12:21:49.975466 5490 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40529
I20260925 12:21:49.975579 5490 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:49.975749 5490 heartbeater.cc:511] Master 127.1.47.190:40529 requested a full tablet report, sending...
I20260925 12:21:49.976332 5300 ts_manager.cc:194] Registered new tserver with Master: 53bdec81de0a4a65b2ce456c316f10a5 (127.1.47.129:43777)
I20260925 12:21:49.977321 5300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:46061
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:50.003054 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:50.087922 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:50.088178 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:50.088220 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:50.089412 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:50.089490 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:50.089536 5495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:50.091091 5495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:50.091384 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:50.091634 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:50.097046 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.097113 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.097059 5503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.097220 5495 server_base.cc:1034] running on GCE node
I20260925 12:21:50.097530 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:50.097849 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:50.099017 5495 hybrid_clock.cc:648] HybridClock initialized: now 1790338910098994 us; error 43 us; skew 500 ppm
Sep 25 12:21:50 dist-test-slave-l9ww krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338910, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:50.101714 5495 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:50.102078 5495 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:50.102155 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:50.102310 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:50.103415 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "e21c667c83ff41e5b38df7efc8921171"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "ae263782542b01b8895705748ad50c41"
server_key_iv: "92747ca579765502d3634bc58b402b1c"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.103804 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "e21c667c83ff41e5b38df7efc8921171"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "ae263782542b01b8895705748ad50c41"
server_key_iv: "92747ca579765502d3634bc58b402b1c"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.105126 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:50.105939 5509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.106168 5495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:50.106276 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "e21c667c83ff41e5b38df7efc8921171"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "ae263782542b01b8895705748ad50c41"
server_key_iv: "92747ca579765502d3634bc58b402b1c"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.106362 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:50.119688 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:50.122368 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:50.122555 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:50.122828 5495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:50.123169 5495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:50.123237 5495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.123301 5495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:50.123327 5495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.129456 5495 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:40631
I20260925 12:21:50.129498 5622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:40631 every 8 connection(s)
I20260925 12:21:50.129804 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:21:50 dist-test-slave-l9ww krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338910, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:50.138603 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5495
I20260925 12:21:50.138847 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:50.139065 1214 external_mini_cluster.cc:1428] Setting key 840c1da87e012b92a37d2f5ea0ff266b
I20260925 12:21:50.140064 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.130736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:45145 (local address 127.1.47.190:40529)
0925 12:21:50.130802 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.130804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.131010 (+ 206us) server_negotiation.cc:407] Connection header received
0925 12:21:50.131504 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.131506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.131517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.131588 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:50.132677 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.133186 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.134224 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.134416 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.134484 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.136831 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:50.136832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:50.136832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:50.136839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:50.138266 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.138797 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.138798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.138798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.138875 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.139198 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.139199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.139200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.139376 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:50.139458 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.139829 (+ 371us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.139975 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:21:50.140249 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.130674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40529 (local address 127.1.47.130:45145)
0925 12:21:50.130848 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.130892 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.131387 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.131647 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.131651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.132024 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:50.132547 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.132554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.133287 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.133288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.134129 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.134131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.134498 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.134499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.134599 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.135111 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:50.135121 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:50.136730 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:50.138361 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:50.138368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:50.138370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:50.138608 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:50.138952 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:50.138953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:50.138953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:50.139069 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:50.139473 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:50.139480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:50.139650 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.139773 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.140004 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260925 12:21:50.140733 5623 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40529
I20260925 12:21:50.140856 5623 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:50.141029 5623 heartbeater.cc:511] Master 127.1.47.190:40529 requested a full tablet report, sending...
I20260925 12:21:50.141479 5300 ts_manager.cc:194] Registered new tserver with Master: e21c667c83ff41e5b38df7efc8921171 (127.1.47.130:40631)
I20260925 12:21:50.141862 5300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:45145
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:50.170912 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:36961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:50.253674 5628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:50.253897 5628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:50.253922 5628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:50.255064 5628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:50.255102 5628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:50.255129 5628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:50.256821 5628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:36961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:50.257023 5628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:50.257202 5628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:50.262511 5633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.262526 5636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.262544 5634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.262565 5628 server_base.cc:1034] running on GCE node
I20260925 12:21:50.262967 5628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:50.263234 5628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:50.264405 5628 hybrid_clock.cc:648] HybridClock initialized: now 1790338910264383 us; error 38 us; skew 500 ppm
Sep 25 12:21:50 dist-test-slave-l9ww krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338910, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:50.267295 5628 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:50.267755 5628 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:50.267832 5628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:50.268006 5628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:50.269147 5628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "bda00588df254c38ba384ff4d030053f"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "71c4c5c592fd67034d240951785ca0af"
server_key_iv: "5d507b029af64f118fa8ff75a88af08b"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.269526 5628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "bda00588df254c38ba384ff4d030053f"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "71c4c5c592fd67034d240951785ca0af"
server_key_iv: "5d507b029af64f118fa8ff75a88af08b"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.270865 5628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:50.271641 5642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.271840 5628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:50.271967 5628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "bda00588df254c38ba384ff4d030053f"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "71c4c5c592fd67034d240951785ca0af"
server_key_iv: "5d507b029af64f118fa8ff75a88af08b"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.272045 5628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:50.290058 5628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:50.292372 5628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:50.292493 5628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:50.292677 5628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:50.293027 5628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:50.293066 5628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.293090 5628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:50.293102 5628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.299089 5628 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:42027
I20260925 12:21:50.299118 5755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:42027 every 8 connection(s)
I20260925 12:21:50.299396 5628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:21:50 dist-test-slave-l9ww krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338910, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:50.306103 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5628
I20260925 12:21:50.306193 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:50.306385 1214 external_mini_cluster.cc:1428] Setting key 5beeefefb8d74d29670e237b52768a85
I20260925 12:21:50.309295 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.300183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:42105 (local address 127.1.47.190:40529)
0925 12:21:50.300236 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.300239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.300501 (+ 262us) server_negotiation.cc:407] Connection header received
0925 12:21:50.300996 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.300997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.301009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.301062 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:50.302027 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.302571 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.303572 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.303788 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.303926 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.306283 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:50.306284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:50.306285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:50.306293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:50.308067 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.308453 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.308453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.308454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.308505 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.308717 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.308717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.308717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.308824 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:50.308874 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.309150 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.309229 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:21:50.309510 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.300165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40529 (local address 127.1.47.131:42105)
0925 12:21:50.300368 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.300407 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.300873 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.301125 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.301128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.301385 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:50.301899 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.301904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.302662 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.302663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.303462 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.303464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.303937 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.303938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.304060 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.304580 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:50.304589 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:50.306160 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:50.308144 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:50.308151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:50.308153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:50.308387 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:50.308561 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:50.308562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:50.308563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:50.308664 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:50.308877 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:50.308881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:50.309002 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.309086 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.309212 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260925 12:21:50.309942 5756 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40529
I20260925 12:21:50.310084 5756 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:50.310254 5756 heartbeater.cc:511] Master 127.1.47.190:40529 requested a full tablet report, sending...
I20260925 12:21:50.310699 5300 ts_manager.cc:194] Registered new tserver with Master: bda00588df254c38ba384ff4d030053f (127.1.47.131:42027)
I20260925 12:21:50.311039 5300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:42105
I20260925 12:21:50.318150 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:21:50.326022 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.319533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.1.47.190:40529)
0925 12:21:50.319613 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.319615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.319676 (+ 61us) server_negotiation.cc:407] Connection header received
0925 12:21:50.319769 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.319770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.319781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.319827 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:50.320436 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.320962 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.321742 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.322048 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.322120 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.323384 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:50.323385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:50.323385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:50.323394 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:50.324804 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.325165 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.325166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.325167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.325221 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:50.325421 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:50.325421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:50.325422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:50.325576 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:50.325731 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.325838 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.325917 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:21:50.327600 5300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:50.329190 5300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:50.335574 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40631 (local address 127.0.0.1:39864)
0925 12:21:50.331649 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.331713 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.331790 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.332197 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.332200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.332203 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:50.332277 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.332279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333258 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.333259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.334437 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.334440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335238 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.335328 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.335342 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.335443 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.335499 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
I20260925 12:21:50.335585 5769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39864 (local address 127.1.47.130:40631)
0925 12:21:50.332030 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.332034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.332044 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:50.332084 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.332085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.332097 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.332190 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:50.332361 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333143 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.334532 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335134 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.335288 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.335461 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.335499 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":105,"threads_started":1}
I20260925 12:21:50.336032 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.1.47.129:43777)
0925 12:21:50.331984 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.331989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.332274 (+ 285us) server_negotiation.cc:407] Connection header received
0925 12:21:50.332338 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.332340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.332354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.332422 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:50.332631 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333683 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.334730 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335457 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335531 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.335586 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.335909 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.335951 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260925 12:21:50.336150 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43777 (local address 127.0.0.1:42970)
0925 12:21:50.332160 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.332169 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.332212 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.332506 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.332507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.332509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:50.332545 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.332546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333763 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.333764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.334646 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.334648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335636 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.335708 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.335713 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.335767 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.335802 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260925 12:21:50.336278 5557 tablet_service.cc:1467] Processing CreateTablet for tablet 87474add05844e0085f1021befa1908d (DEFAULT_TABLE table=test-table [id=d9b5fcb9032c41949e6a03d0661d7410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:50.336535 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:42027 (local address 127.0.0.1:45146)
0925 12:21:50.331984 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.331994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.332078 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.332338 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.332339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.332341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:50.332391 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.332393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333709 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.333710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.334776 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.334779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335994 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.336058 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.336062 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.336381 (+ 319us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.336423 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":88,"threads_started":1}
I20260925 12:21:50.336659 5557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87474add05844e0085f1021befa1908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:50.336836 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.331774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.1.47.131:42027)
0925 12:21:50.331945 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.331951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.332065 (+ 114us) server_negotiation.cc:407] Connection header received
0925 12:21:50.332152 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.332155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.332174 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.332251 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:50.332468 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.333591 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.334859 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.335521 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.335577 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.335962 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.336401 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.336440 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":129,"threads_started":1}
I20260925 12:21:50.336788 5424 tablet_service.cc:1467] Processing CreateTablet for tablet 87474add05844e0085f1021befa1908d (DEFAULT_TABLE table=test-table [id=d9b5fcb9032c41949e6a03d0661d7410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:50.337059 5424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87474add05844e0085f1021befa1908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:50.337383 5690 tablet_service.cc:1467] Processing CreateTablet for tablet 87474add05844e0085f1021befa1908d (DEFAULT_TABLE table=test-table [id=d9b5fcb9032c41949e6a03d0661d7410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:50.337672 5690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87474add05844e0085f1021befa1908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:50.339989 5773 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Bootstrap starting.
I20260925 12:21:50.340207 5774 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Bootstrap starting.
I20260925 12:21:50.340473 5775 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Bootstrap starting.
I20260925 12:21:50.340764 5773 tablet_bootstrap.cc:654] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:50.340859 5774 tablet_bootstrap.cc:654] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:50.341069 5775 tablet_bootstrap.cc:654] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:50.341339 5773 log.cc:824] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:50.341470 5774 log.cc:824] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:50.341665 5775 log.cc:824] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:50.342110 5773 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: No bootstrap required, opened a new log
I20260925 12:21:50.342196 5773 ts_tablet_manager.cc:1397] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 12:21:50.342424 5775 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: No bootstrap required, opened a new log
I20260925 12:21:50.342489 5775 ts_tablet_manager.cc:1397] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:21:50.344173 5773 raft_consensus.cc:348] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.344208 5775 raft_consensus.cc:348] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.344338 5775 raft_consensus.cc:374] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:50.344342 5773 raft_consensus.cc:374] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:50.344360 5775 raft_consensus.cc:729] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda00588df254c38ba384ff4d030053f, State: Initialized, Role: FOLLOWER
I20260925 12:21:50.344391 5773 raft_consensus.cc:729] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21c667c83ff41e5b38df7efc8921171, State: Initialized, Role: FOLLOWER
I20260925 12:21:50.344504 5775 consensus_queue.cc:260] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.344506 5773 consensus_queue.cc:260] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.344789 5773 ts_tablet_manager.cc:1428] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 12:21:50.344798 5775 ts_tablet_manager.cc:1428] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:21:50.344835 5623 heartbeater.cc:503] Master 127.1.47.190:40529 was elected leader, sending a full tablet report...
I20260925 12:21:50.344899 5756 heartbeater.cc:503] Master 127.1.47.190:40529 was elected leader, sending a full tablet report...
I20260925 12:21:50.345263 5774 tablet_bootstrap.cc:492] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: No bootstrap required, opened a new log
I20260925 12:21:50.345335 5774 ts_tablet_manager.cc:1397] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 12:21:50.346937 5774 raft_consensus.cc:348] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.347126 5774 raft_consensus.cc:374] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:50.347189 5774 raft_consensus.cc:729] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bdec81de0a4a65b2ce456c316f10a5, State: Initialized, Role: FOLLOWER
I20260925 12:21:50.347306 5774 consensus_queue.cc:260] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.347605 5774 ts_tablet_manager.cc:1428] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:21:50.347635 5490 heartbeater.cc:503] Master 127.1.47.190:40529 was elected leader, sending a full tablet report...
W20260925 12:21:50.380378 5624 tablet.cc:2369] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:21:50.465435 5491 tablet.cc:2369] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:50.531813 5779 raft_consensus.cc:482] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:50.531975 5779 raft_consensus.cc:504] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.532404 5779 leader_election.cc:288] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bda00588df254c38ba384ff4d030053f (127.1.47.131:42027), 53bdec81de0a4a65b2ce456c316f10a5 (127.1.47.129:43777)
I20260925 12:21:50.535445 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.532649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43777 (local address 127.1.47.130:38945)
0925 12:21:50.532967 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.532978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.533039 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.533177 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.533178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.533180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:50.533249 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.533251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.533874 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.533875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.534669 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.534671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.535203 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.535204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.535275 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.535280 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.535333 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.535374 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":121,"thread_start_us":119,"threads_started":1}
I20260925 12:21:50.535465 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.532740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:38945 (local address 127.1.47.129:43777)
0925 12:21:50.532806 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.532808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.533048 (+ 240us) server_negotiation.cc:407] Connection header received
0925 12:21:50.533088 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.533089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.533102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.533165 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:50.533313 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.533789 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.534769 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.535122 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.535177 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.535184 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.535352 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.535389 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:21:50.535614 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.532649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:42027 (local address 127.1.47.130:35145)
0925 12:21:50.532724 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.532737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.532787 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.533002 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.533003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.533005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:50.533108 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.533111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.533815 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.533816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.534671 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:50.534673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.535358 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.535359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:50.535433 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.535436 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.535496 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.535536 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":68}
I20260925 12:21:50.535650 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.532775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:35145 (local address 127.1.47.131:42027)
0925 12:21:50.532818 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.532819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.532828 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:50.532859 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.532860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.532870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.532977 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:50.533191 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.533722 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.534785 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.535249 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.535323 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.535331 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.535525 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.535570 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:21:50.535861 5444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87474add05844e0085f1021befa1908d" candidate_uuid: "e21c667c83ff41e5b38df7efc8921171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bdec81de0a4a65b2ce456c316f10a5" is_pre_election: true
I20260925 12:21:50.535981 5710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87474add05844e0085f1021befa1908d" candidate_uuid: "e21c667c83ff41e5b38df7efc8921171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda00588df254c38ba384ff4d030053f" is_pre_election: true
I20260925 12:21:50.536048 5444 raft_consensus.cc:2454] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e21c667c83ff41e5b38df7efc8921171 in term 0.
I20260925 12:21:50.536103 5710 raft_consensus.cc:2454] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e21c667c83ff41e5b38df7efc8921171 in term 0.
I20260925 12:21:50.536286 5511 leader_election.cc:302] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bdec81de0a4a65b2ce456c316f10a5, e21c667c83ff41e5b38df7efc8921171; no voters:
I20260925 12:21:50.536414 5779 raft_consensus.cc:2781] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:50.536468 5779 raft_consensus.cc:482] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:50.536491 5779 raft_consensus.cc:3037] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:50.537213 5779 raft_consensus.cc:504] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.537369 5779 leader_election.cc:288] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [CANDIDATE]: Term 1 election: Requested vote from peers bda00588df254c38ba384ff4d030053f (127.1.47.131:42027), 53bdec81de0a4a65b2ce456c316f10a5 (127.1.47.129:43777)
I20260925 12:21:50.537513 5710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87474add05844e0085f1021befa1908d" candidate_uuid: "e21c667c83ff41e5b38df7efc8921171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda00588df254c38ba384ff4d030053f"
I20260925 12:21:50.537514 5444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87474add05844e0085f1021befa1908d" candidate_uuid: "e21c667c83ff41e5b38df7efc8921171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bdec81de0a4a65b2ce456c316f10a5"
I20260925 12:21:50.537593 5444 raft_consensus.cc:3037] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:50.537593 5710 raft_consensus.cc:3037] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:50.538496 5444 raft_consensus.cc:2454] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e21c667c83ff41e5b38df7efc8921171 in term 1.
I20260925 12:21:50.538499 5710 raft_consensus.cc:2454] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e21c667c83ff41e5b38df7efc8921171 in term 1.
I20260925 12:21:50.538702 5511 leader_election.cc:302] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bdec81de0a4a65b2ce456c316f10a5, e21c667c83ff41e5b38df7efc8921171; no voters:
I20260925 12:21:50.538841 5779 raft_consensus.cc:2781] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:50.539036 5779 raft_consensus.cc:686] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 1 LEADER]: Becoming Leader. State: Replica: e21c667c83ff41e5b38df7efc8921171, State: Running, Role: LEADER
I20260925 12:21:50.539142 5779 consensus_queue.cc:237] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } }
I20260925 12:21:50.540048 5300 catalog_manager.cc:5495] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 reported cstate change: term changed from 0 to 1, leader changed from <none> to e21c667c83ff41e5b38df7efc8921171 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "e21c667c83ff41e5b38df7efc8921171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e21c667c83ff41e5b38df7efc8921171" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 } health_report { overall_health: UNKNOWN } } }
W20260925 12:21:50.549954 5757 tablet.cc:2369] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:50.562265 5769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.559958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.1.47.130:40631)
0925 12:21:50.560024 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.560026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.560033 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:50.560067 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.560067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.560086 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.560150 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:50.560286 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.560749 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.561459 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:50.561654 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:50.561712 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:50.561879 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:50.561994 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:50.562059 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.562093 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.562110 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:21:50.564255 5444 raft_consensus.cc:1261] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5 [term 1 FOLLOWER]: Refusing update from remote peer e21c667c83ff41e5b38df7efc8921171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:50.564255 5710 raft_consensus.cc:1261] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 1 FOLLOWER]: Refusing update from remote peer e21c667c83ff41e5b38df7efc8921171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:50.564663 5783 consensus_queue.cc:1035] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53bdec81de0a4a65b2ce456c316f10a5" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:50.564806 5779 consensus_queue.cc:1035] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda00588df254c38ba384ff4d030053f" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:50.567030 5786 mvcc.cc:205] Tried to move back new op lower bound from 7333228177668026368 to 7333228177569001472. Current Snapshot: MvccSnapshot[applied={T|T < 7333228177668026368}]
I20260925 12:21:50.567499 5787 mvcc.cc:205] Tried to move back new op lower bound from 7333228177668026368 to 7333228177569001472. Current Snapshot: MvccSnapshot[applied={T|T < 7333228177668026368}]
I20260925 12:21:50.569788 5298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:50.569904 5298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:50.570722 5298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4f9847afeb14bba9751726747308b86: Sending DeleteTablet for 3 replicas of tablet 87474add05844e0085f1021befa1908d
I20260925 12:21:50.571005 5557 tablet_service.cc:1514] Processing DeleteTablet for tablet 87474add05844e0085f1021befa1908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:50 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:39864
I20260925 12:21:50.571059 5424 tablet_service.cc:1514] Processing DeleteTablet for tablet 87474add05844e0085f1021befa1908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:50 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:42970
I20260925 12:21:50.571200 5690 tablet_service.cc:1514] Processing DeleteTablet for tablet 87474add05844e0085f1021befa1908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:50 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:45146
I20260925 12:21:50.571403 5796 tablet_replica.cc:330] T 87474add05844e0085f1021befa1908d P 53bdec81de0a4a65b2ce456c316f10a5: stopping tablet replica
I20260925 12:21:50.571468 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5362
I20260925 12:21:50.572144 5797 tablet_replica.cc:330] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: stopping tablet replica
I20260925 12:21:50.572338 5797 raft_consensus.cc:2229] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:50.572464 5795 tablet_replica.cc:330] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: stopping tablet replica
I20260925 12:21:50.572556 5795 raft_consensus.cc:2229] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:50.572569 5797 raft_consensus.cc:2258] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:50.572686 5795 raft_consensus.cc:2258] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:50.573099 5795 ts_tablet_manager.cc:1905] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:50.574702 5795 ts_tablet_manager.cc:1918] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:50.574790 5795 log.cc:1196] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/87474add05844e0085f1021befa1908d
I20260925 12:21:50.575068 5795 ts_tablet_manager.cc:1939] T 87474add05844e0085f1021befa1908d P e21c667c83ff41e5b38df7efc8921171: Deleting consensus metadata
I20260925 12:21:50.575594 5285 catalog_manager.cc:4841] TS e21c667c83ff41e5b38df7efc8921171 (127.1.47.130:40631): tablet 87474add05844e0085f1021befa1908d (table test-table [id=d9b5fcb9032c41949e6a03d0661d7410]) successfully deleted
I20260925 12:21:50.576129 5797 ts_tablet_manager.cc:1905] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:50.578001 5797 ts_tablet_manager.cc:1918] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260925 12:21:50.578279 5285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43777 (error 108)
I20260925 12:21:50.578433 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5495
I20260925 12:21:50.579797 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.579629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43777 (local address 127.0.0.1:42974)
0925 12:21:50.579705 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.579753 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 12:21:50.579948 5285 catalog_manager.cc:4584] TS 53bdec81de0a4a65b2ce456c316f10a5 (127.1.47.129:43777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87474add05844e0085f1021befa1908d: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43777: connect: Connection refused (error 111)
I20260925 12:21:50.583181 5797 log.cc:1196] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/87474add05844e0085f1021befa1908d
I20260925 12:21:50.583362 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5628
I20260925 12:21:50.583549 5797 ts_tablet_manager.cc:1939] T 87474add05844e0085f1021befa1908d P bda00588df254c38ba384ff4d030053f: Deleting consensus metadata
I20260925 12:21:50.587265 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5269
I20260925 12:21:50.587466 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.587313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:42027 (local address 127.0.0.1:45162)
0925 12:21:50.587388 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.587423 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.131:42027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
2026-09-25T12:21:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2189 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T12:21:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:50Z Disabled control of system clock
I20260925 12:21:50.614519 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40239
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:33425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:40239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 12:21:50.694618 5802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:50.694835 5802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:50.694856 5802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:50.696074 5802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:50.696116 5802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:50.696130 5802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:50.696142 5802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:50.696205 5802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:50.697528 5802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:40239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:50.697803 5802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:50.698120 5802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:50.702965 5808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.703150 5802 server_base.cc:1034] running on GCE node
W20260925 12:21:50.702961 5810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.702962 5807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.703495 5802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:50.703812 5802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:50.705000 5802 hybrid_clock.cc:648] HybridClock initialized: now 1790338910704981 us; error 39 us; skew 500 ppm
I20260925 12:21:50.706230 5802 webserver.cc:460] Webserver started at http://127.1.47.190:44197/ using document root <none> and password file <none>
I20260925 12:21:50.706483 5802 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:50.706554 5802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:50.706696 5802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:50.707768 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "e80aaeb712df41e68df93db7c1c555c8"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "2608b21b0ab1fe09e7e9a3a932d8a818"
server_key_iv: "0fcea6c1e0bf5689308b35a1c6ec2fd1"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.708204 5802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "e80aaeb712df41e68df93db7c1c555c8"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "2608b21b0ab1fe09e7e9a3a932d8a818"
server_key_iv: "0fcea6c1e0bf5689308b35a1c6ec2fd1"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.709509 5802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:50.710232 5816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.710402 5802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:50.710505 5802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "e80aaeb712df41e68df93db7c1c555c8"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "2608b21b0ab1fe09e7e9a3a932d8a818"
server_key_iv: "0fcea6c1e0bf5689308b35a1c6ec2fd1"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.710585 5802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:50.713999 5802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:50.714270 5802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:50.714409 5802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:50.718158 5802 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:40239
I20260925 12:21:50.718197 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:40239 every 8 connection(s)
I20260925 12:21:50.718523 5802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:50.719146 5869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:50.721845 5869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: Bootstrap starting.
I20260925 12:21:50.722450 5869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:50.722805 5869 log.cc:824] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:50.723562 5869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: No bootstrap required, opened a new log
I20260925 12:21:50.725365 5869 raft_consensus.cc:348] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } }
I20260925 12:21:50.725502 5869 raft_consensus.cc:374] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:50.725525 5869 raft_consensus.cc:729] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80aaeb712df41e68df93db7c1c555c8, State: Initialized, Role: FOLLOWER
I20260925 12:21:50.725690 5869 consensus_queue.cc:260] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } }
I20260925 12:21:50.725764 5869 raft_consensus.cc:388] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:50.725809 5869 raft_consensus.cc:482] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:50.725857 5869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:50.726395 5869 raft_consensus.cc:504] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } }
I20260925 12:21:50.726511 5869 leader_election.cc:302] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e80aaeb712df41e68df93db7c1c555c8; no voters:
I20260925 12:21:50.726709 5869 leader_election.cc:288] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:50.726804 5872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:50.726972 5869 sys_catalog.cc:564] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:50.726971 5872 raft_consensus.cc:686] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [term 1 LEADER]: Becoming Leader. State: Replica: e80aaeb712df41e68df93db7c1c555c8, State: Running, Role: LEADER
I20260925 12:21:50.727115 5872 consensus_queue.cc:237] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } }
I20260925 12:21:50.727504 5873 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e80aaeb712df41e68df93db7c1c555c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } } }
I20260925 12:21:50.727588 5873 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:50.727813 5874 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e80aaeb712df41e68df93db7c1c555c8. Latest consensus state: current_term: 1 leader_uuid: "e80aaeb712df41e68df93db7c1c555c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80aaeb712df41e68df93db7c1c555c8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40239 } } }
I20260925 12:21:50.727932 5874 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:50.728114 5881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:50.728600 5881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:50.728775 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5802
I20260925 12:21:50.728871 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:50.729063 1214 external_mini_cluster.cc:1428] Setting key 0c229831209bd423cdc3898318f28232
I20260925 12:21:50.730427 5881 catalog_manager.cc:1347] Generated new cluster ID: a736bc6c75a74c11a621e79ebaed2edd
I20260925 12:21:50.730477 5881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:50.731594 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.729577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.1.47.190:40239)
0925 12:21:50.729805 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.729814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.729848 (+ 34us) server_negotiation.cc:407] Connection header received
0925 12:21:50.730017 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.730025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.730072 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.730548 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:50.731048 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:50.731051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:50.731051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:50.731054 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:50.731091 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:50.731105 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:50.731349 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.731377 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.731432 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260925 12:21:50.770988 5881 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:50.771183 5881 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:50.773918 5881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: Generated new TSK 0
I20260925 12:21:50.774142 5881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:21:50.781832 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--builtin_ntp_servers=127.1.47.148:33425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 12:21:50.868139 5893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:50.868379 5893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:50.868404 5893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:50.869765 5893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:50.869812 5893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:50.869841 5893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:50.871356 5893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:50.871651 5893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:50.872056 5893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:50.877538 5899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.877550 5898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.877810 5893 server_base.cc:1034] running on GCE node
W20260925 12:21:50.877550 5901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.878122 5893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:50.878407 5893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:50.879590 5893 hybrid_clock.cc:648] HybridClock initialized: now 1790338910879563 us; error 39 us; skew 500 ppm
I20260925 12:21:50.881152 5893 webserver.cc:460] Webserver started at http://127.1.47.129:33793/ using document root <none> and password file <none>
I20260925 12:21:50.881443 5893 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:50.881496 5893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:50.881636 5893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:50.882771 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "cecde138f2b44a23b37e3427897e93e1"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "a2608d9266d55a98de9a51d196164df4"
server_key_iv: "881ecc242326c7469761bf061363c070"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.883150 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "cecde138f2b44a23b37e3427897e93e1"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "a2608d9266d55a98de9a51d196164df4"
server_key_iv: "881ecc242326c7469761bf061363c070"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.884660 5893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:50.885710 5907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.885963 5893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:50.886085 5893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "cecde138f2b44a23b37e3427897e93e1"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "a2608d9266d55a98de9a51d196164df4"
server_key_iv: "881ecc242326c7469761bf061363c070"
server_key_version: "encryptionkey@0"
I20260925 12:21:50.886185 5893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:50.890209 5893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:50.890563 5893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:50.890733 5893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:50.890959 5893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:50.891311 5893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:50.891371 5893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.891417 5893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:50.891446 5893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:50.897842 5893 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:36195
I20260925 12:21:50.897917 6020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:36195 every 8 connection(s)
I20260925 12:21:50.898310 5893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:50.901682 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.899138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:50423 (local address 127.1.47.190:40239)
0925 12:21:50.899205 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:50.899207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:50.899517 (+ 310us) server_negotiation.cc:407] Connection header received
0925 12:21:50.900145 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:50.900146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:50.900154 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:50.900201 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:50.901419 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:50.901421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:50.901421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:50.901426 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:50.901443 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:50.901448 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:50.901509 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:50.901592 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 12:21:50.901610 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:21:50.901902 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:50.899144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40239 (local address 127.1.47.129:50423)
0925 12:21:50.899380 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:50.899422 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:50.900033 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:50.900276 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:50.900280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:50.900289 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:50.901242 (+ 953us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:50.901243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:50.901273 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:50.901304 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:50.901507 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:50.901512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:50.901516 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:50.901591 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 12:21:50.901676 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260925 12:21:50.902417 6021 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40239
I20260925 12:21:50.902509 6021 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:50.902632 6021 heartbeater.cc:511] Master 127.1.47.190:40239 requested a full tablet report, sending...
I20260925 12:21:50.903091 5833 ts_manager.cc:194] Registered new tserver with Master: cecde138f2b44a23b37e3427897e93e1 (127.1.47.129:36195)
I20260925 12:21:50.907125 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 5893
I20260925 12:21:50.907259 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:50.907425 1214 external_mini_cluster.cc:1428] Setting key 884aa7b84cff70b2f4b07bfbbc3c67de
I20260925 12:21:50.908459 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--builtin_ntp_servers=127.1.47.148:33425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 12:21:50.988984 6024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:50.989261 6024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:50.989295 6024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:50.990593 6024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:50.990633 6024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:50.990659 6024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:50.991956 6024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:50.992161 6024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:50.992393 6024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:50.997438 6032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.997501 6030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:50.997454 6029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:50.997748 6024 server_base.cc:1034] running on GCE node
I20260925 12:21:50.998016 6024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:50.998291 6024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:50.999469 6024 hybrid_clock.cc:648] HybridClock initialized: now 1790338910999454 us; error 32 us; skew 500 ppm
I20260925 12:21:51.000941 6024 webserver.cc:460] Webserver started at http://127.1.47.130:41813/ using document root <none> and password file <none>
I20260925 12:21:51.001199 6024 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:51.001276 6024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:51.001420 6024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:51.002498 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "622826f19a7646bea970bf35e86ddd0d"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "f23d4c4b7350989e7821ff571f210680"
server_key_iv: "2507796c1f51d6713edd47acc564723b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.002902 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "622826f19a7646bea970bf35e86ddd0d"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "f23d4c4b7350989e7821ff571f210680"
server_key_iv: "2507796c1f51d6713edd47acc564723b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.004441 6024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:51.005282 6038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.005489 6024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:51.005615 6024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "622826f19a7646bea970bf35e86ddd0d"
format_stamp: "Formatted at 2026-09-25 12:21:50 on dist-test-slave-l9ww"
server_key: "f23d4c4b7350989e7821ff571f210680"
server_key_iv: "2507796c1f51d6713edd47acc564723b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.005697 6024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:51.009096 6024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:51.009351 6024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:51.009490 6024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:51.009706 6024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:51.010000 6024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:51.010056 6024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.010094 6024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:51.010124 6024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.016117 6024 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:45485
I20260925 12:21:51.016157 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:45485 every 8 connection(s)
I20260925 12:21:51.016499 6024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:51.019634 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.017267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:55895 (local address 127.1.47.190:40239)
0925 12:21:51.017339 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.017341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.017568 (+ 227us) server_negotiation.cc:407] Connection header received
0925 12:21:51.018081 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.018082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.018088 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.018137 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.019351 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.019353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.019353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.019358 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.019374 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.019378 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.019431 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.019539 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.019554 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 12:21:51.019847 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.017253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40239 (local address 127.1.47.130:55895)
0925 12:21:51.017447 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.017480 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.017969 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.018208 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.018212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.018221 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.019200 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.019201 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.019228 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.019256 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.019447 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.019451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.019453 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.019549 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.019610 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260925 12:21:51.020385 6152 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40239
I20260925 12:21:51.020457 6152 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:51.020639 6152 heartbeater.cc:511] Master 127.1.47.190:40239 requested a full tablet report, sending...
I20260925 12:21:51.021068 5833 ts_manager.cc:194] Registered new tserver with Master: 622826f19a7646bea970bf35e86ddd0d (127.1.47.130:45485)
I20260925 12:21:51.023677 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6024
I20260925 12:21:51.023799 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:51.024025 1214 external_mini_cluster.cc:1428] Setting key d8176661597ab2b4520bd57d350b2caa
I20260925 12:21:51.025018 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--builtin_ntp_servers=127.1.47.148:33425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 12:21:51.106091 6155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:51.106312 6155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:51.106334 6155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:51.107663 6155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:51.107713 6155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:51.107740 6155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:51.109302 6155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:51.109572 6155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:51.109891 6155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:51.115103 6163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.115109 6160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.115109 6161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.115356 6155 server_base.cc:1034] running on GCE node
I20260925 12:21:51.115599 6155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:51.115818 6155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:51.117003 6155 hybrid_clock.cc:648] HybridClock initialized: now 1790338911116983 us; error 39 us; skew 500 ppm
I20260925 12:21:51.118305 6155 webserver.cc:460] Webserver started at http://127.1.47.131:42187/ using document root <none> and password file <none>
I20260925 12:21:51.118567 6155 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:51.118615 6155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:51.118836 6155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:51.120023 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "bf8f950c41434746850af6ac9dbbf843"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "f4ab7da80e1e11f61e924a90a8e370c1"
server_key_iv: "79b7a9ce8995523cfef5b5d06d888333"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.120398 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "bf8f950c41434746850af6ac9dbbf843"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "f4ab7da80e1e11f61e924a90a8e370c1"
server_key_iv: "79b7a9ce8995523cfef5b5d06d888333"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.121735 6155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:51.122498 6169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.122691 6155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:51.122792 6155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "bf8f950c41434746850af6ac9dbbf843"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "f4ab7da80e1e11f61e924a90a8e370c1"
server_key_iv: "79b7a9ce8995523cfef5b5d06d888333"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.122875 6155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:51.126489 6155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:51.126780 6155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:51.126971 6155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:51.127225 6155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:51.127566 6155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:51.127599 6155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.127620 6155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:51.127662 6155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.133692 6155 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:36767
I20260925 12:21:51.133774 6282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:36767 every 8 connection(s)
I20260925 12:21:51.134114 6155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:51.137264 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.134925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:58841 (local address 127.1.47.190:40239)
0925 12:21:51.134989 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.134991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.135238 (+ 247us) server_negotiation.cc:407] Connection header received
0925 12:21:51.135679 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.135680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.135692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.135750 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.137032 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.137034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.137034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.137039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.137053 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.137058 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.137138 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.137192 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.137206 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:51.137452 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.134916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40239 (local address 127.1.47.131:58841)
0925 12:21:51.135086 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.135128 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.135584 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.135810 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.135813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.135821 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.136852 (+ 1031us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.136852 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.136882 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.136911 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.137129 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.137131 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.137134 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.137188 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.137235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260925 12:21:51.138024 6283 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40239
I20260925 12:21:51.138104 6283 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:51.138281 6283 heartbeater.cc:511] Master 127.1.47.190:40239 requested a full tablet report, sending...
I20260925 12:21:51.138715 5833 ts_manager.cc:194] Registered new tserver with Master: bf8f950c41434746850af6ac9dbbf843 (127.1.47.131:36767)
I20260925 12:21:51.140065 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6155
I20260925 12:21:51.140172 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:51.140337 1214 external_mini_cluster.cc:1428] Setting key de81578224343bdc34b860ba82c95aeb
I20260925 12:21:51.141482 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:21:51.144266 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.143141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.1.47.190:40239)
0925 12:21:51.143196 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.143198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.143270 (+ 72us) server_negotiation.cc:407] Connection header received
0925 12:21:51.143337 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.143338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.143342 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.143381 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.144019 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.144020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.144020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.144024 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.144040 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.144044 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.144104 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.144191 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.144204 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:21:51.146046 5833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:51.147521 5833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:21:51.151381 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.149891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36195 (local address 127.0.0.1:57810)
0925 12:21:51.150199 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.150211 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.150263 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.150436 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.150442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.150448 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.150930 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.150930 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.150944 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.150964 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.151214 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.151219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.151222 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.151290 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.151304 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":106,"threads_started":1}
I20260925 12:21:51.151397 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.149980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.1.47.129:36195)
0925 12:21:51.150249 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.150254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.150267 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:21:51.150329 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.150332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.150341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.150436 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.151070 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.151079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.151079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.151083 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.151117 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.151148 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.151220 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.151307 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.151322 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":116,"threads_started":1}
I20260925 12:21:51.152379 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.149874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:36767 (local address 127.0.0.1:59928)
0925 12:21:51.149990 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.150031 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.150081 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.151097 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.151098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.151101 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.151534 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.151535 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.151543 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.151556 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.152244 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.152245 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.152246 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.152301 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.152315 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260925 12:21:51.152438 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.150254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.1.47.131:36767)
0925 12:21:51.150945 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.150949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.150957 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:51.150977 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.150978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.150984 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.151073 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.152102 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.152105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.152106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.152109 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.152139 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.152154 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.152246 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.152315 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.152333 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":76,"threads_started":1}
I20260925 12:21:51.152490 5955 tablet_service.cc:1467] Processing CreateTablet for tablet 11d4282218b743838a0eef9b4461d9fc (DEFAULT_TABLE table=test-table [id=a2a9797668074730a9af600e9b7e2ff1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:51.152895 5955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d4282218b743838a0eef9b4461d9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.153100 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.150091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:45485 (local address 127.0.0.1:35160)
0925 12:21:51.150383 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.150394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.150438 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.151799 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.151799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.151801 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.152653 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.152653 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.152660 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.152672 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.152930 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.152931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.152931 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.153026 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.153039 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":80,"threads_started":1}
I20260925 12:21:51.153195 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.150207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.1.47.130:45485)
0925 12:21:51.150572 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.150576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.150587 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:51.150611 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.150613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.150624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.151548 (+ 924us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.152765 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.152771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.152772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.152777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.152807 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.152833 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.152896 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.153102 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.153118 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":74,"threads_started":1}
I20260925 12:21:51.155798 6297 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: Bootstrap starting.
I20260925 12:21:51.156503 6297 tablet_bootstrap.cc:654] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.156472 6086 tablet_service.cc:1467] Processing CreateTablet for tablet 11d4282218b743838a0eef9b4461d9fc (DEFAULT_TABLE table=test-table [id=a2a9797668074730a9af600e9b7e2ff1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:51.156472 6217 tablet_service.cc:1467] Processing CreateTablet for tablet 11d4282218b743838a0eef9b4461d9fc (DEFAULT_TABLE table=test-table [id=a2a9797668074730a9af600e9b7e2ff1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:51.156809 6217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d4282218b743838a0eef9b4461d9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.156809 6086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d4282218b743838a0eef9b4461d9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.156840 6297 log.cc:824] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.157616 6297 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: No bootstrap required, opened a new log
I20260925 12:21:51.157693 6297 ts_tablet_manager.cc:1397] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:21:51.159505 6297 raft_consensus.cc:348] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.159682 6297 raft_consensus.cc:374] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.159729 6297 raft_consensus.cc:729] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cecde138f2b44a23b37e3427897e93e1, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.159894 6297 consensus_queue.cc:260] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.160161 6297 ts_tablet_manager.cc:1428] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260925 12:21:51.160286 6021 heartbeater.cc:503] Master 127.1.47.190:40239 was elected leader, sending a full tablet report...
I20260925 12:21:51.160427 6300 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Bootstrap starting.
I20260925 12:21:51.161101 6300 tablet_bootstrap.cc:654] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.161465 6300 log.cc:824] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.161808 6299 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Bootstrap starting.
I20260925 12:21:51.162324 6299 tablet_bootstrap.cc:654] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.162595 6299 log.cc:824] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.163054 6300 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: No bootstrap required, opened a new log
I20260925 12:21:51.163141 6300 ts_tablet_manager.cc:1397] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 12:21:51.163304 6299 tablet_bootstrap.cc:492] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: No bootstrap required, opened a new log
I20260925 12:21:51.163357 6299 ts_tablet_manager.cc:1397] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:51.165246 6300 raft_consensus.cc:348] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.165406 6300 raft_consensus.cc:374] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.165331 6299 raft_consensus.cc:348] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.165452 6300 raft_consensus.cc:729] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8f950c41434746850af6ac9dbbf843, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.165496 6299 raft_consensus.cc:374] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.165539 6299 raft_consensus.cc:729] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622826f19a7646bea970bf35e86ddd0d, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.165575 6300 consensus_queue.cc:260] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.165652 6299 consensus_queue.cc:260] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.165866 6283 heartbeater.cc:503] Master 127.1.47.190:40239 was elected leader, sending a full tablet report...
I20260925 12:21:51.165864 6300 ts_tablet_manager.cc:1428] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:21:51.165905 6299 ts_tablet_manager.cc:1428] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:21:51.165912 6152 heartbeater.cc:503] Master 127.1.47.190:40239 was elected leader, sending a full tablet report...
I20260925 12:21:51.191918 6304 raft_consensus.cc:482] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:51.192101 6304 raft_consensus.cc:504] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.192476 6304 leader_election.cc:288] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cecde138f2b44a23b37e3427897e93e1 (127.1.47.129:36195), 622826f19a7646bea970bf35e86ddd0d (127.1.47.130:45485)
I20260925 12:21:51.193959 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.192764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:41373 (local address 127.1.47.130:45485)
0925 12:21:51.192815 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.192818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.192828 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:51.192858 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.192859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.192864 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.192961 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.193680 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.193693 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.193694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.193699 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.193717 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.193721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.193807 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.193888 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.193902 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:21:51.193959 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.192679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:45485 (local address 127.1.47.131:41373)
0925 12:21:51.192728 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.192740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.192777 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.192966 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.192968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.192970 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.193476 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.193477 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.193485 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.193500 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.193833 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.193833 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.193834 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.193875 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.193889 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 12:21:51.194062 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.192784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:59941 (local address 127.1.47.129:36195)
0925 12:21:51.192828 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.192830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.192989 (+ 159us) server_negotiation.cc:407] Connection header received
0925 12:21:51.193043 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.193044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.193049 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.193100 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.193807 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.193808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.193809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.193813 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.193831 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.193836 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.193892 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.193962 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.193975 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:21:51.194255 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.192679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36195 (local address 127.1.47.131:59941)
0925 12:21:51.192905 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.192913 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.192982 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.193102 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.193103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.193105 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:51.193685 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:21:51.193686 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:51.193693 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:21:51.193705 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:51.193903 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:51.193904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:51.193904 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 12:21:51.193953 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:21:51.193968 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":64,"thread_start_us":120,"threads_started":1}
I20260925 12:21:51.194279 6106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d4282218b743838a0eef9b4461d9fc" candidate_uuid: "bf8f950c41434746850af6ac9dbbf843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622826f19a7646bea970bf35e86ddd0d" is_pre_election: true
I20260925 12:21:51.194427 6106 raft_consensus.cc:2454] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8f950c41434746850af6ac9dbbf843 in term 0.
I20260925 12:21:51.194593 5975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d4282218b743838a0eef9b4461d9fc" candidate_uuid: "bf8f950c41434746850af6ac9dbbf843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cecde138f2b44a23b37e3427897e93e1" is_pre_election: true
I20260925 12:21:51.194753 5975 raft_consensus.cc:2454] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8f950c41434746850af6ac9dbbf843 in term 0.
I20260925 12:21:51.194749 6173 leader_election.cc:302] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622826f19a7646bea970bf35e86ddd0d, bf8f950c41434746850af6ac9dbbf843; no voters:
I20260925 12:21:51.194877 6304 raft_consensus.cc:2781] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:51.194931 6304 raft_consensus.cc:482] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:51.194964 6304 raft_consensus.cc:3037] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.195679 6304 raft_consensus.cc:504] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.195842 6304 leader_election.cc:288] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [CANDIDATE]: Term 1 election: Requested vote from peers cecde138f2b44a23b37e3427897e93e1 (127.1.47.129:36195), 622826f19a7646bea970bf35e86ddd0d (127.1.47.130:45485)
I20260925 12:21:51.196017 6106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d4282218b743838a0eef9b4461d9fc" candidate_uuid: "bf8f950c41434746850af6ac9dbbf843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622826f19a7646bea970bf35e86ddd0d"
I20260925 12:21:51.196019 5975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d4282218b743838a0eef9b4461d9fc" candidate_uuid: "bf8f950c41434746850af6ac9dbbf843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cecde138f2b44a23b37e3427897e93e1"
I20260925 12:21:51.196094 6106 raft_consensus.cc:3037] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.196116 5975 raft_consensus.cc:3037] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.196828 5975 raft_consensus.cc:2454] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8f950c41434746850af6ac9dbbf843 in term 1.
I20260925 12:21:51.196837 6106 raft_consensus.cc:2454] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8f950c41434746850af6ac9dbbf843 in term 1.
I20260925 12:21:51.196988 6173 leader_election.cc:302] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622826f19a7646bea970bf35e86ddd0d, bf8f950c41434746850af6ac9dbbf843; no voters:
I20260925 12:21:51.197085 6304 raft_consensus.cc:2781] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:51.197239 6304 raft_consensus.cc:686] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 1 LEADER]: Becoming Leader. State: Replica: bf8f950c41434746850af6ac9dbbf843, State: Running, Role: LEADER
I20260925 12:21:51.197345 6304 consensus_queue.cc:237] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } }
I20260925 12:21:51.198221 5833 catalog_manager.cc:5495] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf8f950c41434746850af6ac9dbbf843 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "bf8f950c41434746850af6ac9dbbf843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8f950c41434746850af6ac9dbbf843" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:51.213394 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.212358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.1.47.131:36767)
0925 12:21:51.212419 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.212421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.212431 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:51.212471 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.212472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.212486 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.212540 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.213138 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.213140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.213140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.213144 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.213160 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.213164 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.213213 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.213304 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.213317 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:51.215397 6106 raft_consensus.cc:1261] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 1 FOLLOWER]: Refusing update from remote peer bf8f950c41434746850af6ac9dbbf843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:51.215425 5975 raft_consensus.cc:1261] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 1 FOLLOWER]: Refusing update from remote peer bf8f950c41434746850af6ac9dbbf843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:51.215797 6304 consensus_queue.cc:1035] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecde138f2b44a23b37e3427897e93e1" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:51.215962 6307 consensus_queue.cc:1035] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "622826f19a7646bea970bf35e86ddd0d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:51.219069 6311 mvcc.cc:205] Tried to move back new op lower bound from 7333228180335349760 to 7333228180264988672. Current Snapshot: MvccSnapshot[applied={T|T < 7333228180335349760}]
I20260925 12:21:51.220863 6312 mvcc.cc:205] Tried to move back new op lower bound from 7333228180335349760 to 7333228180264988672. Current Snapshot: MvccSnapshot[applied={T|T < 7333228180335349760}]
I20260925 12:21:51.221153 5832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:51.221256 5832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:51.222160 5832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e80aaeb712df41e68df93db7c1c555c8: Sending DeleteTablet for 3 replicas of tablet 11d4282218b743838a0eef9b4461d9fc
I20260925 12:21:51.222582 5955 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d4282218b743838a0eef9b4461d9fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:51 UTC) from {username='slave'} at 127.0.0.1:57810
I20260925 12:21:51.222675 6086 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d4282218b743838a0eef9b4461d9fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:51 UTC) from {username='slave'} at 127.0.0.1:35160
I20260925 12:21:51.222878 6319 tablet_replica.cc:330] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1: stopping tablet replica
I20260925 12:21:51.223029 6319 raft_consensus.cc:2229] T 11d4282218b743838a0eef9b4461d9fc P cecde138f2b44a23b37e3427897e93e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:51.223040 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5893
I20260925 12:21:51.224257 6217 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d4282218b743838a0eef9b4461d9fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:51 UTC) from {username='slave'} at 127.0.0.1:59928
I20260925 12:21:51.224498 6320 tablet_replica.cc:330] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: stopping tablet replica
I20260925 12:21:51.224609 6320 raft_consensus.cc:2229] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:51.224728 6320 raft_consensus.cc:2258] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:51.225085 6321 tablet_replica.cc:330] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: stopping tablet replica
I20260925 12:21:51.225250 6321 raft_consensus.cc:2229] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:51.225404 6321 raft_consensus.cc:2258] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:51.225561 6320 ts_tablet_manager.cc:1905] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:51.226012 6321 ts_tablet_manager.cc:1905] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:51.227572 6320 ts_tablet_manager.cc:1918] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:51.227655 6320 log.cc:1196] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/11d4282218b743838a0eef9b4461d9fc
I20260925 12:21:51.227998 6320 ts_tablet_manager.cc:1939] T 11d4282218b743838a0eef9b4461d9fc P 622826f19a7646bea970bf35e86ddd0d: Deleting consensus metadata
I20260925 12:21:51.228547 6321 ts_tablet_manager.cc:1918] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:51.228685 6321 log.cc:1196] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/11d4282218b743838a0eef9b4461d9fc
I20260925 12:21:51.228772 5820 catalog_manager.cc:4841] TS 622826f19a7646bea970bf35e86ddd0d (127.1.47.130:45485): tablet 11d4282218b743838a0eef9b4461d9fc (table test-table [id=a2a9797668074730a9af600e9b7e2ff1]) successfully deleted
I20260925 12:21:51.229102 6321 ts_tablet_manager.cc:1939] T 11d4282218b743838a0eef9b4461d9fc P bf8f950c41434746850af6ac9dbbf843: Deleting consensus metadata
I20260925 12:21:51.229575 5820 catalog_manager.cc:4841] TS bf8f950c41434746850af6ac9dbbf843 (127.1.47.131:36767): tablet 11d4282218b743838a0eef9b4461d9fc (table test-table [id=a2a9797668074730a9af600e9b7e2ff1]) successfully deleted
W20260925 12:21:51.230084 5817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:36195 (error 108)
I20260925 12:21:51.230643 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.230487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36195 (local address 127.0.0.1:57818)
0925 12:21:51.230554 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.230605 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 12:21:51.230698 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6024
W20260925 12:21:51.232838 5817 catalog_manager.cc:4584] TS cecde138f2b44a23b37e3427897e93e1 (127.1.47.129:36195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 11d4282218b743838a0eef9b4461d9fc: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36195: connect: Connection refused (error 111)
I20260925 12:21:51.236095 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6155
I20260925 12:21:51.240684 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 5802
2026-09-25T12:21:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (652 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 12:21:51.260396 1214 mini_oidc.cc:152] Starting JWKS server
I20260925 12:21:51.261436 1214 webserver.cc:460] Webserver started at https://127.0.0.1:37509/ using document root <none> and password file <none>
I20260925 12:21:51.261518 1214 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 12:21:51.261695 1214 webserver.cc:460] Webserver started at http://127.0.0.1:36457/ using document root <none> and password file <none>
2026-09-25T12:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:51Z Disabled control of system clock
I20260925 12:21:51.269665 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:39395
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:39971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:39395
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37509/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/testchainca.cert with env {}
W20260925 12:21:51.350638 6328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:51.350854 6328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:51.350878 6328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:51.352308 6328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:51.352370 6328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:51.352416 6328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:51.352439 6328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:51.352469 6328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 12:21:51.352495 6328 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 12:21:51.352525 6328 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37509/jwks.json
I20260925 12:21:51.354382 6328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:39395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:39395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37509/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:51.354657 6328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:51.354923 6328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:51.359781 6333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.359822 6336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.359831 6334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.360088 6328 server_base.cc:1034] running on GCE node
I20260925 12:21:51.360368 6328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:51.360697 6328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:51.361852 6328 hybrid_clock.cc:648] HybridClock initialized: now 1790338911361831 us; error 41 us; skew 500 ppm
I20260925 12:21:51.367041 6328 webserver.cc:460] Webserver started at http://127.1.47.190:35403/ using document root <none> and password file <none>
I20260925 12:21:51.367324 6328 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:51.367373 6328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:51.367523 6328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:51.368662 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "4db3812561694e67899ad0117b64e8be"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "a289e1f0de829eeb7316b25c13c36f76"
server_key_iv: "f35dff22bb2e97f7099009bcfb9e786b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.369035 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "4db3812561694e67899ad0117b64e8be"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "a289e1f0de829eeb7316b25c13c36f76"
server_key_iv: "f35dff22bb2e97f7099009bcfb9e786b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.370357 6328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:51.371202 6342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.371428 6328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:51.371527 6328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "4db3812561694e67899ad0117b64e8be"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "a289e1f0de829eeb7316b25c13c36f76"
server_key_iv: "f35dff22bb2e97f7099009bcfb9e786b"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.371609 6328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:51.407351 6328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:51.407720 6328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:51.407925 6328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:51.411684 6328 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:39395
I20260925 12:21:51.411741 6394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:39395 every 8 connection(s)
I20260925 12:21:51.412138 6328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:51.413149 6395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.414613 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6328
I20260925 12:21:51.414747 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:51.414932 1214 external_mini_cluster.cc:1428] Setting key 88a3cbdaf4a8b4c1593c987639e9455c
I20260925 12:21:51.415700 6395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be: Bootstrap starting.
I20260925 12:21:51.416406 6395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.416745 6395 log.cc:824] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.417534 6395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be: No bootstrap required, opened a new log
I20260925 12:21:51.420020 6395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } }
I20260925 12:21:51.420203 6395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.420256 6395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db3812561694e67899ad0117b64e8be, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.420396 6395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } }
I20260925 12:21:51.420502 6395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:51.420547 6395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:51.420595 6395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.421087 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.415477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.1.47.190:39395)
0925 12:21:51.415654 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.415662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.415692 (+ 30us) server_negotiation.cc:407] Connection header received
0925 12:21:51.416180 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.416188 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.416264 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.416382 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.416653 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.418060 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.419136 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.419525 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.419612 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.420334 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.420340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.420340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.420552 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.420596 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.420609 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.420706 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.420799 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.420891 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260925 12:21:51.421247 6395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } }
I20260925 12:21:51.421350 6395 leader_election.cc:302] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4db3812561694e67899ad0117b64e8be; no voters:
I20260925 12:21:51.421499 6395 leader_election.cc:288] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:51.421555 6400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:51.421718 6400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [term 1 LEADER]: Becoming Leader. State: Replica: 4db3812561694e67899ad0117b64e8be, State: Running, Role: LEADER
I20260925 12:21:51.421784 6395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:51.421813 6400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } }
I20260925 12:21:51.422021 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:21:51.422137 6402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4db3812561694e67899ad0117b64e8be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } } }
I20260925 12:21:51.422215 6402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:51.422379 6401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4db3812561694e67899ad0117b64e8be. Latest consensus state: current_term: 1 leader_uuid: "4db3812561694e67899ad0117b64e8be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4db3812561694e67899ad0117b64e8be" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39395 } } }
I20260925 12:21:51.422431 6401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:51.422765 6407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:51.423216 6407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:51.424873 6407 catalog_manager.cc:1347] Generated new cluster ID: 6be5ee4ce6654d5d9fd03377faa5a686
I20260925 12:21:51.424924 6407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:51.477087 6407 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:51.477654 6407 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:51.483573 6407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4db3812561694e67899ad0117b64e8be: Generated new TSK 0
I20260925 12:21:51.483825 6407 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 12:21:51.510938 6425 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 12:21:51.511062 6425 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260925 12:21:51.511561 6424 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:51.490648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:39395 (local address 127.0.0.1:37808)
0925 12:21:51.490860 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.490873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.490947 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.491143 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.491143 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.491145 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 12:21:51.491210 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.491212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.491835 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.491836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.492725 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.492729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.493080 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.493081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.493154 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.493179 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 12:21:51.511218 (+ 18039us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 12:21:51.511317 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.47.190:39395: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
I20260925 12:21:51.511904 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.490742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.1.47.190:39395)
0925 12:21:51.490789 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.490791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.490940 (+ 149us) server_negotiation.cc:407] Connection header received
0925 12:21:51.491037 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.491038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.491050 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.491108 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0925 12:21:51.491270 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.491755 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.492811 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.492995 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.493047 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.493288 (+ 241us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 12:21:51.511045 (+ 17757us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 12:21:51.511817 (+ 772us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 12:21:51.511955 6398 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.47.190:39395: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37509/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 12:21:51.512370 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6328
2026-09-25T12:21:51Z chronyd exiting
I20260925 12:21:51.548678 1214 test_util.cc:175] -----------------------------------------------
I20260925 12:21:51.548764 1214 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790338888974512-1214-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (289 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 12:21:51.549559 1214 mini_oidc.cc:152] Starting JWKS server
I20260925 12:21:51.550305 1214 webserver.cc:460] Webserver started at https://127.0.0.1:33051/ using document root <none> and password file <none>
I20260925 12:21:51.550348 1214 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 12:21:51.550544 1214 webserver.cc:460] Webserver started at http://127.0.0.1:40721/ using document root <none> and password file <none>
2026-09-25T12:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:51Z Disabled control of system clock
I20260925 12:21:51.557950 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36737
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36737
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33051/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/testchainca.cert with env {}
W20260925 12:21:51.638989 6431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:51.639259 6431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:51.639298 6431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:51.640599 6431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:51.640673 6431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:51.640695 6431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:51.640717 6431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:51.640743 6431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 12:21:51.640762 6431 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 12:21:51.640782 6431 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33051/jwks.json
I20260925 12:21:51.642351 6431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33051/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:51.642652 6431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:51.642881 6431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:51.647861 6436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.647972 6431 server_base.cc:1034] running on GCE node
W20260925 12:21:51.647943 6437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.647922 6439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.648360 6431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:51.648654 6431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:51.649813 6431 hybrid_clock.cc:648] HybridClock initialized: now 1790338911649798 us; error 30 us; skew 500 ppm
I20260925 12:21:51.651135 6431 webserver.cc:460] Webserver started at http://127.1.47.190:39563/ using document root <none> and password file <none>
I20260925 12:21:51.651361 6431 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:51.651405 6431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:51.651496 6431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:51.652616 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "d4b9f07646d54338aa8d4899ed6f494b"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "836343454639211815eba4ed29e262a2"
server_key_iv: "c1c2e7ab6d7eae8347810ee9db9c612e"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.652920 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "d4b9f07646d54338aa8d4899ed6f494b"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "836343454639211815eba4ed29e262a2"
server_key_iv: "c1c2e7ab6d7eae8347810ee9db9c612e"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.654137 6431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:51.654826 6445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.655010 6431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:51.655107 6431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "d4b9f07646d54338aa8d4899ed6f494b"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "836343454639211815eba4ed29e262a2"
server_key_iv: "c1c2e7ab6d7eae8347810ee9db9c612e"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.655164 6431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:51.671787 6431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:51.672178 6431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:51.672297 6431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:51.676054 6431 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:36737
I20260925 12:21:51.676151 6497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:36737 every 8 connection(s)
I20260925 12:21:51.676486 6431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:51.677501 6498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.679795 6498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: Bootstrap starting.
I20260925 12:21:51.680415 6498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.680711 6498 log.cc:824] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.681402 6498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: No bootstrap required, opened a new log
I20260925 12:21:51.682473 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6431
I20260925 12:21:51.682561 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:51.682739 1214 external_mini_cluster.cc:1428] Setting key a949696f6c130b323fc18ec703c84888
I20260925 12:21:51.683357 6498 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } }
I20260925 12:21:51.683547 6498 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.683584 6498 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b9f07646d54338aa8d4899ed6f494b, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.683717 6498 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } }
I20260925 12:21:51.683792 6498 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:51.683820 6498 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:51.683894 6498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.684624 6498 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } }
I20260925 12:21:51.684742 6498 leader_election.cc:302] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4b9f07646d54338aa8d4899ed6f494b; no voters:
I20260925 12:21:51.684885 6498 leader_election.cc:288] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:51.684971 6503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:51.685107 6503 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [term 1 LEADER]: Becoming Leader. State: Replica: d4b9f07646d54338aa8d4899ed6f494b, State: Running, Role: LEADER
I20260925 12:21:51.685153 6498 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:51.685210 6503 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } }
I20260925 12:21:51.685518 6503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4b9f07646d54338aa8d4899ed6f494b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } } }
I20260925 12:21:51.685612 6503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:51.685730 6503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b9f07646d54338aa8d4899ed6f494b. Latest consensus state: current_term: 1 leader_uuid: "d4b9f07646d54338aa8d4899ed6f494b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b9f07646d54338aa8d4899ed6f494b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36737 } } }
I20260925 12:21:51.685784 6503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b [sys.catalog]: This master's current role is: LEADER
W20260925 12:21:51.686903 6518 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:51.686959 6518 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:51.687002 6510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:51.687166 6510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:51.689121 6510 catalog_manager.cc:1347] Generated new cluster ID: 8e82cae8209544a6b6849f5713947af6
I20260925 12:21:51.689175 6510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:51.689628 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.683321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.1.47.190:36737)
0925 12:21:51.683563 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.683573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.683606 (+ 33us) server_negotiation.cc:407] Connection header received
0925 12:21:51.683721 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.683729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.683800 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.683964 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.684200 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.686019 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.687180 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.687782 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.687935 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.688888 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.688892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.688893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.689102 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.689145 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.689170 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.689239 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.689348 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.689424 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
I20260925 12:21:51.690536 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:21:51.703646 6510 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:51.704589 6510 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:51.717042 6510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4b9f07646d54338aa8d4899ed6f494b: Generated new TSK 0
I20260925 12:21:51.717298 6510 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 12:21:51.750855 6528 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 12:21:51.750952 6528 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 12:21:51.751444 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.729178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.1.47.190:36737)
0925 12:21:51.729253 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.729255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.729373 (+ 118us) server_negotiation.cc:407] Connection header received
0925 12:21:51.729464 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.729465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.729479 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.729581 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0925 12:21:51.729734 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.730269 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.731093 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.731326 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.731411 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.731626 (+ 215us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 12:21:51.751130 (+ 19504us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 12:21:51.751353 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55414: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 12:21:51.751516 6502 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55414: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 12:21:51.751529 6527 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:51.729109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36737 (local address 127.0.0.1:55414)
0925 12:21:51.729306 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.729315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.729393 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.729580 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.729581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.729583 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 12:21:51.729657 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.729659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.730366 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.730367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.730996 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.730997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.731424 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.731425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.731484 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.731489 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 12:21:51.751287 (+ 19798us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 12:21:51.751411 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.47.190:36737: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260925 12:21:51.751830 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6431
2026-09-25T12:21:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 12:21:51.789722 1214 mini_oidc.cc:152] Starting JWKS server
I20260925 12:21:51.790386 1214 webserver.cc:460] Webserver started at https://127.0.0.1:42633/ using document root <none> and password file <none>
I20260925 12:21:51.790426 1214 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 12:21:51.790571 1214 webserver.cc:460] Webserver started at http://127.0.0.1:35451/ using document root <none> and password file <none>
2026-09-25T12:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:51Z Disabled control of system clock
I20260925 12:21:51.798441 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36265
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36265
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42633/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 12:21:51.882905 6534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:51.883159 6534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:51.883200 6534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:51.884449 6534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:51.884532 6534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:51.884557 6534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:51.884584 6534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:51.884614 6534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 12:21:51.884634 6534 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 12:21:51.884654 6534 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42633/jwks.json
I20260925 12:21:51.886063 6534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42633/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:51.886348 6534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:51.886665 6534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:51.891716 6542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.891745 6534 server_base.cc:1034] running on GCE node
W20260925 12:21:51.891707 6539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:51.891716 6540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:51.892308 6534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:51.892624 6534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:51.893769 6534 hybrid_clock.cc:648] HybridClock initialized: now 1790338911893749 us; error 38 us; skew 500 ppm
I20260925 12:21:51.895126 6534 webserver.cc:460] Webserver started at http://127.1.47.190:37819/ using document root <none> and password file <none>
I20260925 12:21:51.895386 6534 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:51.895463 6534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:51.895632 6534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:51.896823 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "00ddccdeecd949c9a03e4d56d1b818dd"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "e37fe8f934cd9566c61fdc5f63bd6e66"
server_key_iv: "ed7b4e8460020960a3fbe0c169d1a9ef"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.897202 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "00ddccdeecd949c9a03e4d56d1b818dd"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "e37fe8f934cd9566c61fdc5f63bd6e66"
server_key_iv: "ed7b4e8460020960a3fbe0c169d1a9ef"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.898504 6534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:51.899238 6548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:51.899412 6534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:51.899513 6534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "00ddccdeecd949c9a03e4d56d1b818dd"
format_stamp: "Formatted at 2026-09-25 12:21:51 on dist-test-slave-l9ww"
server_key: "e37fe8f934cd9566c61fdc5f63bd6e66"
server_key_iv: "ed7b4e8460020960a3fbe0c169d1a9ef"
server_key_version: "encryptionkey@0"
I20260925 12:21:51.899621 6534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:51.908288 6534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:51.908602 6534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:51.908762 6534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:51.912690 6534 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:36265
I20260925 12:21:51.912729 6600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:36265 every 8 connection(s)
I20260925 12:21:51.913113 6534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:51.913805 6601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:51.916764 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: Bootstrap starting.
I20260925 12:21:51.917431 6601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:51.917747 6601 log.cc:824] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:51.918568 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: No bootstrap required, opened a new log
I20260925 12:21:51.920488 6601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } }
I20260925 12:21:51.920637 6601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:51.920662 6601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ddccdeecd949c9a03e4d56d1b818dd, State: Initialized, Role: FOLLOWER
I20260925 12:21:51.920819 6601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } }
I20260925 12:21:51.920903 6601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:51.920929 6601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:51.920989 6601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:51.921588 6601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } }
I20260925 12:21:51.921679 6601 leader_election.cc:302] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ddccdeecd949c9a03e4d56d1b818dd; no voters:
I20260925 12:21:51.921916 6601 leader_election.cc:288] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:51.921974 6604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:51.922183 6604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [term 1 LEADER]: Becoming Leader. State: Replica: 00ddccdeecd949c9a03e4d56d1b818dd, State: Running, Role: LEADER
I20260925 12:21:51.922235 6601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:51.922318 6604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } }
I20260925 12:21:51.922853 6606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ddccdeecd949c9a03e4d56d1b818dd. Latest consensus state: current_term: 1 leader_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } } }
I20260925 12:21:51.923116 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6534
I20260925 12:21:51.923226 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:51.923430 1214 external_mini_cluster.cc:1428] Setting key c955c2d31ee7bf4cec35f6754997444c
I20260925 12:21:51.923851 6606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:51.923101 6605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ddccdeecd949c9a03e4d56d1b818dd" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36265 } } }
I20260925 12:21:51.924436 6605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd [sys.catalog]: This master's current role is: LEADER
W20260925 12:21:51.924563 6619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:51.924625 6619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:51.924680 6620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:51.924878 6620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:51.926471 6620 catalog_manager.cc:1347] Generated new cluster ID: 70e763dbab204d3397bea4a889e43c8c
I20260925 12:21:51.926556 6620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:51.933058 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.924292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.1.47.190:36265)
0925 12:21:51.924594 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.924602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.924657 (+ 55us) server_negotiation.cc:407] Connection header received
0925 12:21:51.925492 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.925499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.925559 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.925719 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:51.925981 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.929002 (+ 3021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.930351 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.931056 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.931183 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.932125 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:51.932130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:51.932130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:51.932363 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:51.932409 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:51.932424 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:51.932493 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:51.932626 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 12:21:51.932814 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":68,"threads_started":1}
I20260925 12:21:51.934065 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:21:51.948832 6620 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:51.949434 6620 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:51.953832 6620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ddccdeecd949c9a03e4d56d1b818dd: Generated new TSK 0
I20260925 12:21:51.954053 6620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:21:51.965721 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:51.962208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.1.47.190:36265)
0925 12:21:51.962278 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:51.962280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:51.962392 (+ 112us) server_negotiation.cc:407] Connection header received
0925 12:21:51.962475 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:51.962475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:51.962488 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:51.962557 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0925 12:21:51.962700 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.963204 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.964066 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.964278 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.964367 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.964563 (+ 196us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 12:21:51.965499 (+ 936us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42633/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 12:21:51.965629 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53140: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42633/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 12:21:51.965791 6623 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53140: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42633/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 12:21:51.965834 6630 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:51.962148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36265 (local address 127.0.0.1:53140)
0925 12:21:51.962321 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:51.962331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:51.962410 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:51.962568 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:51.962569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:51.962571 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 12:21:51.962622 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.962624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.963293 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.963294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.963983 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:51.963984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:51.964374 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:51.964375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:51.964432 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:51.964437 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 12:21:51.965618 (+ 1181us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42633/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 12:21:51.965734 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.47.190:36265: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42633/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260925 12:21:51.966226 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6534
E20260925 12:21:51.980374 6631 webserver.cc:533] Webserver: sslize failed:
E20260925 12:21:51.980459 6631 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T12:21:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T12:21:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:52Z Disabled control of system clock
I20260925 12:21:52.011740 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45929
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:45929
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/testchainca.cert with env {}
W20260925 12:21:52.096362 6635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:52.096589 6635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:52.096612 6635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:52.097880 6635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:52.097929 6635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:52.097942 6635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:52.097955 6635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:52.098018 6635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 12:21:52.098037 6635 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 12:21:52.098066 6635 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 12:21:52.099632 6635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:45929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:52.099944 6635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:52.100186 6635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:52.105440 6640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:52.105449 6641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:52.105463 6643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:52.105738 6635 server_base.cc:1034] running on GCE node
I20260925 12:21:52.106035 6635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:52.106351 6635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:52.107530 6635 hybrid_clock.cc:648] HybridClock initialized: now 1790338912107515 us; error 33 us; skew 500 ppm
I20260925 12:21:52.108894 6635 webserver.cc:460] Webserver started at http://127.1.47.190:46271/ using document root <none> and password file <none>
I20260925 12:21:52.109155 6635 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:52.109227 6635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:52.109380 6635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:52.110476 6635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "f38153ff7d3d487da416d5dc5aca5b67"
format_stamp: "Formatted at 2026-09-25 12:21:52 on dist-test-slave-l9ww"
server_key: "1ba925f24a47158dd2ea6890771f5f29"
server_key_iv: "f31180b2501353c356ba66cc6a8750f2"
server_key_version: "encryptionkey@0"
I20260925 12:21:52.110852 6635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "f38153ff7d3d487da416d5dc5aca5b67"
format_stamp: "Formatted at 2026-09-25 12:21:52 on dist-test-slave-l9ww"
server_key: "1ba925f24a47158dd2ea6890771f5f29"
server_key_iv: "f31180b2501353c356ba66cc6a8750f2"
server_key_version: "encryptionkey@0"
I20260925 12:21:52.112234 6635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:52.112967 6649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:52.113139 6635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:52.113246 6635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "f38153ff7d3d487da416d5dc5aca5b67"
format_stamp: "Formatted at 2026-09-25 12:21:52 on dist-test-slave-l9ww"
server_key: "1ba925f24a47158dd2ea6890771f5f29"
server_key_iv: "f31180b2501353c356ba66cc6a8750f2"
server_key_version: "encryptionkey@0"
I20260925 12:21:52.113322 6635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:52.139148 6635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:52.139528 6635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:52.139714 6635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:52.143530 6635 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:45929
I20260925 12:21:52.143575 6701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:45929 every 8 connection(s)
I20260925 12:21:52.144018 6635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:52.145018 6702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:52.146538 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6635
I20260925 12:21:52.146693 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:52.146903 1214 external_mini_cluster.cc:1428] Setting key 31830fd8606d3fa7f8c042ba5d357503
I20260925 12:21:52.147967 6702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67: Bootstrap starting.
I20260925 12:21:52.148695 6702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:52.149013 6702 log.cc:824] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:52.149798 6702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67: No bootstrap required, opened a new log
I20260925 12:21:52.152060 6702 raft_consensus.cc:348] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } }
I20260925 12:21:52.152235 6702 raft_consensus.cc:374] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:52.152269 6702 raft_consensus.cc:729] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38153ff7d3d487da416d5dc5aca5b67, State: Initialized, Role: FOLLOWER
I20260925 12:21:52.152411 6702 consensus_queue.cc:260] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } }
I20260925 12:21:52.152498 6702 raft_consensus.cc:388] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:52.152537 6702 raft_consensus.cc:482] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:52.152585 6702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:52.153393 6702 raft_consensus.cc:504] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } }
I20260925 12:21:52.153496 6702 leader_election.cc:302] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f38153ff7d3d487da416d5dc5aca5b67; no voters:
I20260925 12:21:52.153564 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:52.147563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.1.47.190:45929)
0925 12:21:52.147813 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:52.147823 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:52.147883 (+ 60us) server_negotiation.cc:407] Connection header received
0925 12:21:52.148475 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:52.148484 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:52.148568 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:52.148732 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:52.148991 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.150483 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.151305 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.152026 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.152161 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:52.152856 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:52.152860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:52.152860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:21:52.153079 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:52.153122 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:21:52.153138 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:52.153206 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:52.153279 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 12:21:52.153361 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260925 12:21:52.153669 6702 leader_election.cc:288] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:52.153872 6708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:52.153913 6702 sys_catalog.cc:564] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:52.154033 6708 raft_consensus.cc:686] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [term 1 LEADER]: Becoming Leader. State: Replica: f38153ff7d3d487da416d5dc5aca5b67, State: Running, Role: LEADER
I20260925 12:21:52.154121 6708 consensus_queue.cc:237] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } }
I20260925 12:21:52.154430 6708 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f38153ff7d3d487da416d5dc5aca5b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } } }
I20260925 12:21:52.154505 6708 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:52.154525 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:21:52.154647 6708 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f38153ff7d3d487da416d5dc5aca5b67. Latest consensus state: current_term: 1 leader_uuid: "f38153ff7d3d487da416d5dc5aca5b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38153ff7d3d487da416d5dc5aca5b67" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45929 } } }
I20260925 12:21:52.154706 6708 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:52.154790 6716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:52.155314 6716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:52.157186 6716 catalog_manager.cc:1347] Generated new cluster ID: 251daeb2fc66478aa6a389ef6eb72b94
I20260925 12:21:52.157245 6716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:52.170624 6716 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:52.171166 6716 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:52.177479 6716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f38153ff7d3d487da416d5dc5aca5b67: Generated new TSK 0
I20260925 12:21:52.177687 6716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:21:52.216013 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:52.188030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.1.47.190:45929)
0925 12:21:52.188085 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:52.188088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:52.188205 (+ 117us) server_negotiation.cc:407] Connection header received
0925 12:21:52.188289 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:52.188290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:52.188300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:52.188363 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0925 12:21:52.188663 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.189162 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.190302 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.190499 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.190553 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:52.190747 (+ 194us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 12:21:52.215675 (+ 24928us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 12:21:52.215952 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59524: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 12:21:52.216106 6705 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59524: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 12:21:52.216161 6731 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:52.187932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:45929 (local address 127.0.0.1:59524)
0925 12:21:52.188121 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:52.188135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:52.188212 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:52.188405 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:52.188406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:52.188408 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 12:21:52.188572 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:52.188577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.189245 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.189246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:52.190207 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:52.190209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:52.190583 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:52.190584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:52.190647 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:52.190652 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 12:21:52.215903 (+ 25251us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 12:21:52.216052 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.47.190:45929: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260925 12:21:52.216681 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6635
2026-09-25T12:21:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (230 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:52 dist-test-slave-l9ww krb5kdc[6743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:52 dist-test-slave-l9ww krb5kdc[6743](info): set up 2 sockets
Sep 25 12:21:52 dist-test-slave-l9ww krb5kdc[6743](info): commencing operation
krb5kdc: starting...
W20260925 12:21:53.432224 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:21:53.559728 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40445
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:40445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:53.641032 6756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:53.641266 6756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:53.641290 6756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:53.642504 6756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:53.642562 6756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:53.642577 6756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:53.642590 6756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:53.642602 6756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:53.644207 6756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:40445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:40445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:53.644488 6756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:53.644733 6756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:53.649616 6762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:53.649626 6761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:53.649682 6756 server_base.cc:1034] running on GCE node
W20260925 12:21:53.649629 6764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:53.650115 6756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:53.650408 6756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:53.651577 6756 hybrid_clock.cc:648] HybridClock initialized: now 1790338913651555 us; error 39 us; skew 500 ppm
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:53.654345 6756 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:21:53.654827 6756 webserver.cc:460] Webserver started at http://127.1.47.190:43137/ using document root <none> and password file <none>
I20260925 12:21:53.655100 6756 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:53.655150 6756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:53.655323 6756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:53.656513 6756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "a9bd5c549b2048cb956c446a58352099"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "677edc1b974c0dc824236a51f537824c"
server_key_iv: "bd0033a1ae1956891235a66879e549de"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.656893 6756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "a9bd5c549b2048cb956c446a58352099"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "677edc1b974c0dc824236a51f537824c"
server_key_iv: "bd0033a1ae1956891235a66879e549de"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.658315 6756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:53.659114 6771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:53.659313 6756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:53.659420 6756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "a9bd5c549b2048cb956c446a58352099"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "677edc1b974c0dc824236a51f537824c"
server_key_iv: "bd0033a1ae1956891235a66879e549de"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.659503 6756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:53.680984 6756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:53.683946 6756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:53.684160 6756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:53.688772 6756 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:40445
I20260925 12:21:53.688819 6823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:40445 every 8 connection(s)
I20260925 12:21:53.689253 6756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:53.690172 6824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:53.692570 6824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: Bootstrap starting.
I20260925 12:21:53.693113 6824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:53.693353 6824 log.cc:824] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:53.694070 6824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: No bootstrap required, opened a new log
I20260925 12:21:53.694892 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6756
I20260925 12:21:53.695015 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:53.695211 1214 external_mini_cluster.cc:1428] Setting key 4d54f631bd6627e20e09407bdf1da866
I20260925 12:21:53.696224 6824 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } }
I20260925 12:21:53.696400 6824 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:53.696440 6824 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bd5c549b2048cb956c446a58352099, State: Initialized, Role: FOLLOWER
I20260925 12:21:53.696583 6824 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } }
I20260925 12:21:53.696666 6824 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:53.696689 6824 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:53.696750 6824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:53.697857 6824 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } }
I20260925 12:21:53.697973 6824 leader_election.cc:302] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9bd5c549b2048cb956c446a58352099; no voters:
I20260925 12:21:53.698127 6824 leader_election.cc:288] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:53.698215 6829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:53.698354 6829 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [term 1 LEADER]: Becoming Leader. State: Replica: a9bd5c549b2048cb956c446a58352099, State: Running, Role: LEADER
I20260925 12:21:53.698367 6824 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:53.698451 6829 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } }
I20260925 12:21:53.698905 6829 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bd5c549b2048cb956c446a58352099. Latest consensus state: current_term: 1 leader_uuid: "a9bd5c549b2048cb956c446a58352099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } } }
I20260925 12:21:53.698976 6829 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:53.700045 6830 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9bd5c549b2048cb956c446a58352099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9bd5c549b2048cb956c446a58352099" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 40445 } } }
I20260925 12:21:53.700281 6830 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099 [sys.catalog]: This master's current role is: LEADER
W20260925 12:21:53.700322 6845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:53.700378 6845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:53.700434 6838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:53.700591 6838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:53.702280 6838 catalog_manager.cc:1347] Generated new cluster ID: 79fb3d4bcc224f6aa35bfadcf4cd1de8
I20260925 12:21:53.702342 6838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:53.707453 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:53.695808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.1.47.190:40445)
0925 12:21:53.696042 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:53.696053 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:53.696093 (+ 40us) server_negotiation.cc:407] Connection header received
0925 12:21:53.696229 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:53.696237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:53.696296 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:53.696453 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:53.697346 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.698792 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.699783 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.700503 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.700627 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:53.703472 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:53.703476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:53.703476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:53.703489 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:53.705558 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:53.705964 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:53.705967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:53.705968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:53.706084 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:53.706252 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:53.706253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:53.706253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:53.706541 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:53.706683 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:53.707105 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 12:21:53.707241 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260925 12:21:53.713388 6838 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:53.713987 6838 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:53.716320 6838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9bd5c549b2048cb956c446a58352099: Generated new TSK 0
I20260925 12:21:53.716516 6838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:21:53.747139 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:53.829557 6850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:53.829757 6850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:53.829780 6850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:53.830966 6850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:53.831008 6850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:53.831034 6850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:53.832621 6850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:53.832818 6850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:53.832998 6850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:53.837942 6856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:53.837951 6855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:53.837949 6858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:53.838024 6850 server_base.cc:1034] running on GCE node
I20260925 12:21:53.838352 6850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:53.838616 6850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:53.839797 6850 hybrid_clock.cc:648] HybridClock initialized: now 1790338913839801 us; error 46 us; skew 500 ppm
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:53.842314 6850 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:21:53.842763 6850 webserver.cc:460] Webserver started at http://127.1.47.129:45025/ using document root <none> and password file <none>
I20260925 12:21:53.843009 6850 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:53.843087 6850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:53.843235 6850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:53.844352 6850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "b02bc72bd6424539a00e7ed6769cfbff"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "7bd35ac675b2b4f8a71d9968af48e224"
server_key_iv: "e1e5f18e023c8ee03eb077bc228d581e"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.844743 6850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "b02bc72bd6424539a00e7ed6769cfbff"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "7bd35ac675b2b4f8a71d9968af48e224"
server_key_iv: "e1e5f18e023c8ee03eb077bc228d581e"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.846107 6850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:53.846815 6865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:53.846978 6850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:53.847088 6850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "b02bc72bd6424539a00e7ed6769cfbff"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "7bd35ac675b2b4f8a71d9968af48e224"
server_key_iv: "e1e5f18e023c8ee03eb077bc228d581e"
server_key_version: "encryptionkey@0"
I20260925 12:21:53.847160 6850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:53.854688 6850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:53.856818 6850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:53.856989 6850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:53.857228 6850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:53.857542 6850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:53.857599 6850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:53.857642 6850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:53.857668 6850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:53.864871 6850 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:44659
I20260925 12:21:53.865017 6978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:44659 every 8 connection(s)
I20260925 12:21:53.865314 6850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:21:53.872141 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6850
I20260925 12:21:53.872256 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:53.872469 1214 external_mini_cluster.cc:1428] Setting key 51f970ec5f989ed28d37b3428562c80e
I20260925 12:21:53.876868 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:53.866174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40445 (local address 127.1.47.129:56995)
0925 12:21:53.866345 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:53.866388 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:53.867314 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:53.867602 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:53.867606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:53.867928 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:53.868158 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:53.868164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.869013 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.869014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:53.869851 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:53.869854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.870312 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.870313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:53.870457 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:53.871018 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:53.871026 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:53.872903 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:53.875054 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:53.875064 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:53.875067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:53.875494 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:53.875767 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:53.875768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:53.875769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:53.875962 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:53.876258 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:53.876264 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:53.876442 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 12:21:53.876542 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 12:21:53.876684 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260925 12:21:53.876874 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:53.866220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:56995 (local address 127.1.47.190:40445)
0925 12:21:53.866268 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:53.866270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:53.866502 (+ 232us) server_negotiation.cc:407] Connection header received
0925 12:21:53.867416 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:53.867417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:53.867432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:53.867541 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:53.868286 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.868897 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.869960 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:53.870194 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:53.870278 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:53.873004 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:53.873005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:53.873005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:53.873014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:53.874950 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:53.875614 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:53.875615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:53.875616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:53.875678 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:53.876044 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:53.876045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:53.876045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:53.876176 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:53.876234 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:53.876628 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 12:21:53.876734 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:21:53.877460 6979 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40445
I20260925 12:21:53.877605 6979 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:53.877826 6979 heartbeater.cc:511] Master 127.1.47.190:40445 requested a full tablet report, sending...
I20260925 12:21:53.878441 6788 ts_manager.cc:194] Registered new tserver with Master: b02bc72bd6424539a00e7ed6769cfbff (127.1.47.129:44659)
I20260925 12:21:53.879446 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:56995
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:21:53.902869 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:53.989797 6984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:53.990010 6984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:53.990032 6984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:53.991117 6984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:53.991148 6984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:53.991174 6984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:53.992578 6984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:53.992771 6984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:53.992942 6984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:53.997823 6992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:53.997838 6990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:53.997838 6989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:53.998046 6984 server_base.cc:1034] running on GCE node
I20260925 12:21:53.998245 6984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:53.998499 6984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:53.999651 6984 hybrid_clock.cc:648] HybridClock initialized: now 1790338913999636 us; error 30 us; skew 500 ppm
Sep 25 12:21:53 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.002588 6984 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:21:54.003021 6984 webserver.cc:460] Webserver started at http://127.1.47.130:45023/ using document root <none> and password file <none>
I20260925 12:21:54.003262 6984 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:54.003332 6984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:54.003485 6984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:54.004611 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "c13e5040945546549af29973a6fe9253"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "0e0a3a6c8c729186067998a1b195309c"
server_key_iv: "ff95d9b76f3571fb4e1a55e921c0956b"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.004977 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "c13e5040945546549af29973a6fe9253"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "0e0a3a6c8c729186067998a1b195309c"
server_key_iv: "ff95d9b76f3571fb4e1a55e921c0956b"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.006306 6984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 12:21:54.007128 6999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.007357 6984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:54.007458 6984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "c13e5040945546549af29973a6fe9253"
format_stamp: "Formatted at 2026-09-25 12:21:53 on dist-test-slave-l9ww"
server_key: "0e0a3a6c8c729186067998a1b195309c"
server_key_iv: "ff95d9b76f3571fb4e1a55e921c0956b"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.007567 6984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:54.036015 6984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:54.038326 6984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:54.038506 6984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:54.038776 6984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:54.039101 6984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:54.039161 6984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.039206 6984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:54.039228 6984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.045094 6984 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:46429
I20260925 12:21:54.045239 7112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:46429 every 8 connection(s)
I20260925 12:21:54.045521 6984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:54.049391 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 6984
I20260925 12:21:54.049474 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:54.049695 1214 external_mini_cluster.cc:1428] Setting key 24201046a658bbac2c53b28b9bbf1ab6
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:54.057120 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.046362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:36747 (local address 127.1.47.190:40445)
0925 12:21:54.046484 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.046486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.046625 (+ 139us) server_negotiation.cc:407] Connection header received
0925 12:21:54.047578 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.047579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.047589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.047652 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.048398 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.048921 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.050605 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.050921 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.050997 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.053949 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:54.053951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:54.053951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:54.053960 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:54.055525 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.056093 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.056094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.056095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.056163 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.056431 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.056432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.056432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.056556 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:54.056641 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:54.056931 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 12:21:54.057025 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 12:21:54.057343 7115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.046337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40445 (local address 127.1.47.130:36747)
0925 12:21:54.046487 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:54.046538 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:54.047458 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:54.047728 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:54.047731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:54.048087 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:54.048291 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:54.048296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.049039 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.049040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:54.050477 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:54.050480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.051013 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.051014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:54.051133 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.051788 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:54.051798 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:54.053727 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:54.055660 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:54.055668 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:54.055670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:54.055996 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:54.056234 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:54.056235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:54.056235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:54.056356 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:54.056631 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:54.056636 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:54.056792 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 12:21:54.056897 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 12:21:54.057077 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
I20260925 12:21:54.057873 7113 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40445
I20260925 12:21:54.057984 7113 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:54.058194 7113 heartbeater.cc:511] Master 127.1.47.190:40445 requested a full tablet report, sending...
I20260925 12:21:54.058531 6788 ts_manager.cc:194] Registered new tserver with Master: c13e5040945546549af29973a6fe9253 (127.1.47.130:46429)
I20260925 12:21:54.058871 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:36747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:21:54.076023 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:54.159957 7118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:54.160215 7118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:54.160257 7118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:54.161468 7118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:54.161550 7118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:54.161597 7118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:54.163295 7118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:40445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:54.163574 7118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:54.163825 7118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:54.168861 7126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:54.168911 7124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:54.168861 7123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:54.169081 7118 server_base.cc:1034] running on GCE node
I20260925 12:21:54.169355 7118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:54.169612 7118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:54.170771 7118 hybrid_clock.cc:648] HybridClock initialized: now 1790338914170751 us; error 54 us; skew 500 ppm
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.173579 7118 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:21:54.174098 7118 webserver.cc:460] Webserver started at http://127.1.47.131:36925/ using document root <none> and password file <none>
I20260925 12:21:54.174357 7118 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:54.174432 7118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:54.174585 7118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:54.175695 7118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "b9c70bed9b4f4f98ac4e1eda2ed98468"
format_stamp: "Formatted at 2026-09-25 12:21:54 on dist-test-slave-l9ww"
server_key: "4335404c114e27e95f5c483f61355aa5"
server_key_iv: "c3ffb0b1d5c791451ecfb6d90a945f5a"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.176102 7118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "b9c70bed9b4f4f98ac4e1eda2ed98468"
format_stamp: "Formatted at 2026-09-25 12:21:54 on dist-test-slave-l9ww"
server_key: "4335404c114e27e95f5c483f61355aa5"
server_key_iv: "c3ffb0b1d5c791451ecfb6d90a945f5a"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.177632 7118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:54.178423 7133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.178632 7118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:54.178735 7118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "b9c70bed9b4f4f98ac4e1eda2ed98468"
format_stamp: "Formatted at 2026-09-25 12:21:54 on dist-test-slave-l9ww"
server_key: "4335404c114e27e95f5c483f61355aa5"
server_key_iv: "c3ffb0b1d5c791451ecfb6d90a945f5a"
server_key_version: "encryptionkey@0"
I20260925 12:21:54.178815 7118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:54.215286 7118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:54.217670 7118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:54.217852 7118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:54.218115 7118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:54.218443 7118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:54.218501 7118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.218575 7118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:54.218604 7118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:54.224435 7118 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:36959
I20260925 12:21:54.224491 7246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:36959 every 8 connection(s)
I20260925 12:21:54.224885 7118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:21:54.232056 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7118
I20260925 12:21:54.232146 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:54.232340 1214 external_mini_cluster.cc:1428] Setting key 691f6a663b640dc3757662154b1f708f
I20260925 12:21:54.235297 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.225690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:55811 (local address 127.1.47.190:40445)
0925 12:21:54.225770 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.225773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.226016 (+ 243us) server_negotiation.cc:407] Connection header received
0925 12:21:54.226907 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.226909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.226919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.226992 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.227603 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.228213 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.229251 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.229468 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.229549 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.232098 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:54.232100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:54.232100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:54.232109 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:54.233814 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.234278 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.234279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.234280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.234341 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.234620 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.234622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.234622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.234752 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:54.234832 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:54.235114 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 12:21:54.235199 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 12:21:54.235498 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.225693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:40445 (local address 127.1.47.131:55811)
0925 12:21:54.225852 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:54.225895 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:54.226808 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:54.227039 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:54.227042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:54.227308 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:54.227498 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:54.227502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.228294 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.228295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:54.229103 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:54.229106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.229560 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.229561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:54.229667 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.230203 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:54.230211 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:54.231976 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:54.233924 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:54.233932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:54.233935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:54.234193 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:54.234418 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:54.234419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:54.234420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:54.234549 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:54.234822 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:54.234829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:54.234979 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 12:21:54.235095 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 12:21:54.235265 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260925 12:21:54.236008 7247 heartbeater.cc:347] Connected to a master server at 127.1.47.190:40445
I20260925 12:21:54.236106 7247 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:54.236307 7247 heartbeater.cc:511] Master 127.1.47.190:40445 requested a full tablet report, sending...
I20260925 12:21:54.236761 6788 ts_manager.cc:194] Registered new tserver with Master: b9c70bed9b4f4f98ac4e1eda2ed98468 (127.1.47.131:36959)
I20260925 12:21:54.237210 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:55811
I20260925 12:21:54.244597 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/test-admin.keytab.
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.262944 1214 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 12:21:54.265005 7254 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:54.264014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.1.47.129:44659)
0925 12:21:54.264228 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.264232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.264243 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:54.264270 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.264273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.264289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.264367 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.264817 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33160: BlockingRecv error: recv got EOF from 127.0.0.1:33160 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":136,"threads_started":1}
W20260925 12:21:54.265046 1214 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc' of type FILE
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.266161 1214 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.267108 1214 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:54.275254 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.267727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.1.47.129:44659)
0925 12:21:54.267786 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.267788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.267890 (+ 102us) server_negotiation.cc:407] Connection header received
0925 12:21:54.268000 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.268001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.268011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.268062 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.268542 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.269262 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.270126 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.270460 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.270517 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.272741 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:54.272749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:54.272749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:54.272757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:54.274275 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.274595 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.274598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.274598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.274650 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.274811 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.274811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.274811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.274936 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:54.275049 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:54.275115 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 12:21:54.275176 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 12:21:54.277421 7254 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:54.276828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.1.47.129:44659)
0925 12:21:54.276883 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.276885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.276914 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:21:54.276992 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.276993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.277002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.277045 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.277377 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33176: BlockingRecv error: recv got EOF from 127.0.0.1:33176 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 12:21:54.277573 1214 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc' of type FILE
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.278779 1214 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.279744 1214 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:54.287240 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.280356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.1.47.129:44659)
0925 12:21:54.280421 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.280422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.280435 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:21:54.280521 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.280521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.280529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.280577 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.280949 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.281519 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.282272 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.282497 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.282565 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.284760 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:54.284762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:54.284762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:54.284770 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:54.286115 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.286478 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.286479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.286480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.286536 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.286736 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.286737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.286737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.286841 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:54.286951 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:54.287053 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 12:21:54.287131 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 12:21:54.288780 7254 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:21:54.288289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.1.47.129:44659)
0925 12:21:54.288325 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.288327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.288371 (+ 44us) server_negotiation.cc:407] Connection header received
0925 12:21:54.288422 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.288422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.288429 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.288473 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.288712 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33194: BlockingRecv error: recv got EOF from 127.0.0.1:33194 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 12:21:54.288898 1214 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790338888974512-1214-0/krb5kdc/krb5cc' of type FILE
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.289880 1214 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:54.290787 1214 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[6743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:21:54.297910 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:54.291336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.1.47.129:44659)
0925 12:21:54.291397 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:54.291399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:54.291412 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:21:54.291469 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:54.291470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:54.291479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:54.291536 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:54.291971 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.292428 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.293089 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:54.293268 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:54.293324 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:54.295283 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:54.295285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:54.295285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:54.295291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:54.296926 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.297222 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.297223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.297223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.297271 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:54.297430 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:54.297431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:54.297431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:54.297518 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:54.297607 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:54.297709 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 12:21:54.297762 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:21:54.298601 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6850
I20260925 12:21:54.303527 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6984
I20260925 12:21:54.308298 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7118
I20260925 12:21:54.312835 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 6756
2026-09-25T12:21:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1921 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[7267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[7267](info): set up 2 sockets
Sep 25 12:21:54 dist-test-slave-l9ww krb5kdc[7267](info): commencing operation
krb5kdc: starting...
W20260925 12:21:55.376183 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:21:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:21:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab.
Entry for principal oryx/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab.
I20260925 12:21:55.495808 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37841
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab
--principal=oryx/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:55.577783 7280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:55.578011 7280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:55.578034 7280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:55.579147 7280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:55.579185 7280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:55.579198 7280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:55.579211 7280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:55.579223 7280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:55.580678 7280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab
--principal=oryx/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:55.580890 7280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:55.581067 7280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:55.586062 7286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:55.586081 7285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:55.586266 7280 server_base.cc:1034] running on GCE node
W20260925 12:21:55.586081 7288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:55.586612 7280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:55.586895 7280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:55.588066 7280 hybrid_clock.cc:648] HybridClock initialized: now 1790338915588051 us; error 31 us; skew 500 ppm
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:55.590816 7280 init.cc:377] Logged in from keytab as oryx/127.1.47.190@KRBTEST.COM (short username oryx)
I20260925 12:21:55.591267 7280 webserver.cc:460] Webserver started at http://127.1.47.190:44213/ using document root <none> and password file <none>
I20260925 12:21:55.591499 7280 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:55.591544 7280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:55.591686 7280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:55.592867 7280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "273fd950106a4234ad34dea2981ba481"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "27d0a7a808777a6dcacf0587c12d1ff6"
server_key_iv: "bfc48a0234f119cd729e10d09bb4accf"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.593250 7280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "273fd950106a4234ad34dea2981ba481"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "27d0a7a808777a6dcacf0587c12d1ff6"
server_key_iv: "bfc48a0234f119cd729e10d09bb4accf"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.594708 7280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:21:55.595465 7295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:55.595686 7280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:55.595782 7280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "273fd950106a4234ad34dea2981ba481"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "27d0a7a808777a6dcacf0587c12d1ff6"
server_key_iv: "bfc48a0234f119cd729e10d09bb4accf"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.595841 7280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:55.612987 7280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:55.615435 7280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:55.615556 7280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:55.619349 7280 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37841
I20260925 12:21:55.619382 7347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37841 every 8 connection(s)
I20260925 12:21:55.619758 7280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:21:55.620807 7348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:55.621174 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7280
I20260925 12:21:55.621287 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:21:55.621481 1214 external_mini_cluster.cc:1428] Setting key 0dfa8d82225d5047e0e52fadeb0735dc
I20260925 12:21:55.623958 7348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Bootstrap starting.
I20260925 12:21:55.624557 7348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:55.624890 7348 log.cc:824] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:55.625667 7348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: No bootstrap required, opened a new log
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:55.628243 7348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } }
I20260925 12:21:55.628450 7348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:55.628497 7348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273fd950106a4234ad34dea2981ba481, State: Initialized, Role: FOLLOWER
I20260925 12:21:55.628638 7348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } }
I20260925 12:21:55.628732 7348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:21:55.628767 7348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:21:55.628813 7348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:55.629580 7348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } }
I20260925 12:21:55.629731 7348 leader_election.cc:302] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 273fd950106a4234ad34dea2981ba481; no voters:
I20260925 12:21:55.629918 7348 leader_election.cc:288] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:21:55.629987 7353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:55.630129 7353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 LEADER]: Becoming Leader. State: Replica: 273fd950106a4234ad34dea2981ba481, State: Running, Role: LEADER
I20260925 12:21:55.630182 7348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:55.630226 7353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } }
I20260925 12:21:55.630532 7353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273fd950106a4234ad34dea2981ba481. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } }
I20260925 12:21:55.630646 7353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:55.630535 7355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } }
I20260925 12:21:55.630787 7355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:55.630954 7357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:21:55.631526 7357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:21:55.632933 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:55.622018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.1.47.190:37841)
0925 12:21:55.622195 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:55.622203 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:55.622228 (+ 25us) server_negotiation.cc:407] Connection header received
0925 12:21:55.622637 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:55.622642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:55.622689 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:55.622807 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:55.623580 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.624539 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.625520 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.625959 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.626049 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:55.628755 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:55.628760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:55.628760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:55.628773 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:55.630773 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:55.631210 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:55.631213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:55.631214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:55.631317 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:55.631558 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:55.631558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:55.631559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:55.631922 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:55.632523 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:55.632592 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 12:21:55.632728 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260925 12:21:55.636574 7357 catalog_manager.cc:1347] Generated new cluster ID: 1869eaf3bc064663b5fcc677050f4381
I20260925 12:21:55.636642 7357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:21:55.646652 7357 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:21:55.647238 7357 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:21:55.651997 7357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Generated new TSK 0
I20260925 12:21:55.652207 7357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
I20260925 12:21:55.683540 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:55.766075 7374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:55.766299 7374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:55.766323 7374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:55.767510 7374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:55.767561 7374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:55.767589 7374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:55.769126 7374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:55.769394 7374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:55.769656 7374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:55.774698 7380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:55.774698 7382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:55.774803 7379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:55.775161 7374 server_base.cc:1034] running on GCE node
I20260925 12:21:55.775345 7374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:55.775583 7374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:55.776746 7374 hybrid_clock.cc:648] HybridClock initialized: now 1790338915776730 us; error 28 us; skew 500 ppm
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:55.779210 7374 init.cc:377] Logged in from keytab as oryx/127.1.47.129@KRBTEST.COM (short username oryx)
I20260925 12:21:55.779654 7374 webserver.cc:460] Webserver started at http://127.1.47.129:34627/ using document root <none> and password file <none>
I20260925 12:21:55.779942 7374 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:55.780009 7374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:55.780166 7374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:55.781230 7374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "a584399904fb4e319b34991da3a8317f"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "d865b4e0e64a60ff7ecff0ed63f521cf"
server_key_iv: "7a2512a2f18be68cc25695545fea9402"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.781594 7374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "a584399904fb4e319b34991da3a8317f"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "d865b4e0e64a60ff7ecff0ed63f521cf"
server_key_iv: "7a2512a2f18be68cc25695545fea9402"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.783136 7374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:55.783962 7389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:55.784143 7374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:55.784242 7374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "a584399904fb4e319b34991da3a8317f"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "d865b4e0e64a60ff7ecff0ed63f521cf"
server_key_iv: "7a2512a2f18be68cc25695545fea9402"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.784327 7374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:55.842335 7374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:55.844378 7374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:55.844544 7374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:55.844791 7374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:55.845129 7374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:55.845189 7374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:55.845233 7374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:55.845261 7374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:55.851223 7374 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43517
I20260925 12:21:55.851266 7502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43517 every 8 connection(s)
I20260925 12:21:55.851631 7374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:55.859252 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7374
I20260925 12:21:55.859380 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:21:55.859608 1214 external_mini_cluster.cc:1428] Setting key f24f9ecacc604ad554e5dac749df0be5
I20260925 12:21:55.862407 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:55.852366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:34445)
0925 12:21:55.852535 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:55.852582 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:55.853083 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:55.853342 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:55.853345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:55.853941 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:55.854150 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:55.854154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.854973 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.854974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:55.856224 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:55.856227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.856636 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.856637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:55.856773 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:55.857409 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:55.857419 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:55.858972 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:55.860842 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:55.860850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:55.860853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:55.861197 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:55.861396 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:55.861396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:55.861397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:55.861581 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:55.861893 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:55.861899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:55.862027 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 12:21:55.862116 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 12:21:55.862241 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260925 12:21:55.862416 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:55.852409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:34445 (local address 127.1.47.190:37841)
0925 12:21:55.852452 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:55.852454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:55.852682 (+ 228us) server_negotiation.cc:407] Connection header received
0925 12:21:55.853183 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:55.853185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:55.853196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:55.853260 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:55.854249 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.854862 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.856308 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:55.856528 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:55.856589 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:55.859063 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:55.859064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:55.859065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:55.859072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:55.860755 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:55.861272 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:55.861273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:55.861273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:55.861323 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:55.861659 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:55.861660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:55.861660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:55.861816 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:55.861889 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:55.862200 (+ 311us) server_negotiation.cc:299] Negotiation successful
0925 12:21:55.862296 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:21:55.862953 7503 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:55.863060 7503 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:55.863271 7503 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:55.864004 7312 ts_manager.cc:194] Registered new tserver with Master: a584399904fb4e319b34991da3a8317f (127.1.47.129:43517)
I20260925 12:21:55.864722 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:34445
WARNING: no policy specified for oryx/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
I20260925 12:21:55.887415 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:55.967594 7508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:55.967875 7508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:55.967942 7508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:55.969147 7508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:55.969226 7508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:55.969273 7508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:55.970753 7508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:55.971052 7508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:55.971306 7508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:55.976279 7513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:55.976310 7516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:55.976346 7514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:55.976593 7508 server_base.cc:1034] running on GCE node
I20260925 12:21:55.976826 7508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:55.977038 7508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:55.978240 7508 hybrid_clock.cc:648] HybridClock initialized: now 1790338915978236 us; error 43 us; skew 500 ppm
Sep 25 12:21:55 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:55.981421 7508 init.cc:377] Logged in from keytab as oryx/127.1.47.130@KRBTEST.COM (short username oryx)
I20260925 12:21:55.981966 7508 webserver.cc:460] Webserver started at http://127.1.47.130:42903/ using document root <none> and password file <none>
I20260925 12:21:55.982192 7508 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:55.982242 7508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:55.982349 7508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:55.983582 7508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "872ec0ce067c43e087553da603ca6e02"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "c19047c0512da3e9d8fdd7d0bf5485d9"
server_key_iv: "535ef35a98e9df4f1833ee6376b73702"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.984030 7508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "872ec0ce067c43e087553da603ca6e02"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "c19047c0512da3e9d8fdd7d0bf5485d9"
server_key_iv: "535ef35a98e9df4f1833ee6376b73702"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.985474 7508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:55.986229 7523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:55.986400 7508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:55.986480 7508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "872ec0ce067c43e087553da603ca6e02"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "c19047c0512da3e9d8fdd7d0bf5485d9"
server_key_iv: "535ef35a98e9df4f1833ee6376b73702"
server_key_version: "encryptionkey@0"
I20260925 12:21:55.986534 7508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.018294 7508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.020741 7508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:56.020864 7508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.021075 7508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:56.021430 7508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:56.021468 7508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.021515 7508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:56.021543 7508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.027305 7508 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:38601
I20260925 12:21:56.027348 7636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:38601 every 8 connection(s)
I20260925 12:21:56.027685 7508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:21:56.032806 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7508
I20260925 12:21:56.032929 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:21:56.033161 1214 external_mini_cluster.cc:1428] Setting key ebba6dea7b0789c3f2d7fdfa957eaff3
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:56.038314 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.028432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:37319 (local address 127.1.47.190:37841)
0925 12:21:56.028555 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.028557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.028726 (+ 169us) server_negotiation.cc:407] Connection header received
0925 12:21:56.029267 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.029269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.029281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.029333 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.030353 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.030970 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.032100 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.032311 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.032394 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.034845 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.034847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.034848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.034855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.036637 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.037186 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.037187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.037188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.037249 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.037551 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.037551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.037552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.037668 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.037719 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.038125 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.038246 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 12:21:56.038559 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.028405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:37319)
0925 12:21:56.028593 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.028638 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.029158 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.029413 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.029417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.030054 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.030251 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.030255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.031065 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.031066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.032007 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.032010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.032404 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.032405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.032541 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.033181 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.033192 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.034741 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.036737 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.036745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.036748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.037091 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.037325 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.037325 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.037326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.037485 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.037732 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.037738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.037931 (+ 193us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.038068 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.038269 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260925 12:21:56.039094 7637 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:56.039213 7637 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:56.039433 7637 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:56.039919 7312 ts_manager.cc:194] Registered new tserver with Master: 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:21:56.040315 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:37319
WARNING: no policy specified for oryx/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
I20260925 12:21:56.062929 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:56.162252 7642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:56.162469 7642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:56.162494 7642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:56.163618 7642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:56.163656 7642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:56.163682 7642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:56.165165 7642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:56.165385 7642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:56.165578 7642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.170687 7647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.170687 7650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.170770 7648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.170877 7642 server_base.cc:1034] running on GCE node
I20260925 12:21:56.171072 7642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.171348 7642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.172509 7642 hybrid_clock.cc:648] HybridClock initialized: now 1790338916172499 us; error 29 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.174930 7642 init.cc:377] Logged in from keytab as oryx/127.1.47.131@KRBTEST.COM (short username oryx)
I20260925 12:21:56.175482 7642 webserver.cc:460] Webserver started at http://127.1.47.131:38031/ using document root <none> and password file <none>
I20260925 12:21:56.175727 7642 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.175796 7642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.175971 7642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:56.177040 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "53d3e915dee14c3c96ff541f6cd61fa2"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "821070ab9eda24269f97af26cc1fbf4e"
server_key_iv: "b0eabf47201c92a00714541660950461"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.177465 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "53d3e915dee14c3c96ff541f6cd61fa2"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "821070ab9eda24269f97af26cc1fbf4e"
server_key_iv: "b0eabf47201c92a00714541660950461"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.179122 7642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:56.179975 7657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.180370 7642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:56.180474 7642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "53d3e915dee14c3c96ff541f6cd61fa2"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "821070ab9eda24269f97af26cc1fbf4e"
server_key_iv: "b0eabf47201c92a00714541660950461"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.180544 7642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.225140 7642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.227059 7642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:56.227173 7642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.227358 7642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:56.227617 7642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:56.227648 7642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.227670 7642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:56.227684 7642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.234027 7642 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:45425
I20260925 12:21:56.234098 7770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:45425 every 8 connection(s)
I20260925 12:21:56.234422 7642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:56.238696 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7642
I20260925 12:21:56.238782 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:21:56.238965 1214 external_mini_cluster.cc:1428] Setting key a83a5a81b4f00e0cb5bd850ce6359564
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:56.243629 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.235245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:36245 (local address 127.1.47.190:37841)
0925 12:21:56.235312 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.235314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.235553 (+ 239us) server_negotiation.cc:407] Connection header received
0925 12:21:56.236044 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.236045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.236055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.236118 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.237154 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.237664 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.238650 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.238951 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.239028 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.241278 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.241279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.241280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.241286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.242487 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.242871 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.242872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.242872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.242922 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.243119 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.243120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.243120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.243215 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.243263 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.243505 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.243564 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 12:21:56.243846 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.235216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:36245)
0925 12:21:56.235378 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.235421 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.235952 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.236188 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.236192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.236857 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.237062 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.237067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.237748 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.237749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.238566 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.238568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.239054 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.239055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.239201 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.239766 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.239775 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.241194 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.242577 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.242582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.242584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.242807 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.242975 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.242975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.242975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.243068 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.243266 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.243270 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.243381 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.243462 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.243577 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260925 12:21:56.244336 7771 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:56.244451 7771 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:56.244647 7771 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:56.244995 7312 ts_manager.cc:194] Registered new tserver with Master: 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
I20260925 12:21:56.245369 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:36245
I20260925 12:21:56.250682 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.189.keytab.
Entry for principal oryx/127.1.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.189.keytab.
I20260925 12:21:56.277082 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:39049
--webserver_interface=127.1.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.189.keytab
--principal=oryx/127.1.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37841,127.1.47.189:39049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:56.358464 7776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:56.358716 7776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:56.358753 7776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:56.359931 7776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:56.360001 7776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:56.360024 7776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:56.360044 7776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:56.360076 7776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:56.361569 7776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37841,127.1.47.189:39049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.189.keytab
--principal=oryx/127.1.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:39049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
W20260925 12:21:56.361721 7776 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.47.190:37841,127.1.47.189:39049'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 12:21:56.361881 7776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:56.362118 7776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.367115 7781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.367228 7782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.367115 7784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.367378 7776 server_base.cc:1034] running on GCE node
I20260925 12:21:56.367646 7776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.368011 7776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.369175 7776 hybrid_clock.cc:648] HybridClock initialized: now 1790338916369161 us; error 29 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.375211 7776 init.cc:377] Logged in from keytab as oryx/127.1.47.189@KRBTEST.COM (short username oryx)
I20260925 12:21:56.375756 7776 webserver.cc:460] Webserver started at http://127.1.47.189:39781/ using document root <none> and password file <none>
I20260925 12:21:56.376044 7776 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.376113 7776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.376284 7776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:56.377338 7776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/instance:
uuid: "a5ee8a1711044aea950df8a6599c351d"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "2b474d629edea60b98ae77ac95c3c40e"
server_key_iv: "99f7fbf5e681087d0a3c1603e4490c49"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.377710 7776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal/instance:
uuid: "a5ee8a1711044aea950df8a6599c351d"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "2b474d629edea60b98ae77ac95c3c40e"
server_key_iv: "99f7fbf5e681087d0a3c1603e4490c49"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.379094 7776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:21:56.380021 7791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.380194 7776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:21:56.380297 7776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "a5ee8a1711044aea950df8a6599c351d"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "2b474d629edea60b98ae77ac95c3c40e"
server_key_iv: "99f7fbf5e681087d0a3c1603e4490c49"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.380406 7776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.391985 7776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.393860 7776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:56.394032 7776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.397845 7776 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.189:39049
I20260925 12:21:56.397883 7843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.189:39049 every 8 connection(s)
I20260925 12:21:56.398298 7776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
I20260925 12:21:56.399386 7844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:56.400938 7844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:21:56.402364 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7776
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.189@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:56.410367 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.401609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.1.47.190:37841)
0925 12:21:56.401687 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.401690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.401857 (+ 167us) server_negotiation.cc:407] Connection header received
0925 12:21:56.402330 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.402332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.402346 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.402422 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.403177 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.403675 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.404701 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.404896 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.404986 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.407223 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.407225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.407225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.407233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.408900 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.409373 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.409374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.409374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.409437 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.409692 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.409693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.409693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.409846 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.409924 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.410195 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.410275 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 12:21:56.410600 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.401560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44138)
0925 12:21:56.401739 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.401776 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.402216 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.402538 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.402542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.402790 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.403074 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.403080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.403782 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.403783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.404594 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.404596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.405012 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.405013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.405134 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.405642 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.405651 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.407124 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.409026 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.409033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.409035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.409295 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.409512 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.409513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.409514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.409617 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.409915 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.409920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.410059 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.410162 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.410314 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":16,"thread_start_us":98,"threads_started":1}
I20260925 12:21:56.411247 7844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:21:56.412235 7848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.402862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.1.47.189:39049)
0925 12:21:56.403016 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.403021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.403045 (+ 24us) server_negotiation.cc:407] Connection header received
0925 12:21:56.403068 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.403071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.403090 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.403182 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.403771 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.404670 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.405430 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.405859 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.405927 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.409082 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.409088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.409089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.409095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.410757 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.411202 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.411204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.411205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.411266 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.411475 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.411476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.411476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.411654 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.411821 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.411980 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.412042 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.189@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:56.418339 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.411488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38612)
0925 12:21:56.411528 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.411538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.411569 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.411808 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.411809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.411989 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.412043 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.412045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.412905 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.412906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.413386 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.413388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.413723 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.413724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.413778 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.414093 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.414097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.415165 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.417185 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.417186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.417187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.417353 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.417567 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.417567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.417567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.417618 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.417954 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.417956 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.418005 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.418085 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.418195 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 12:21:56.418459 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.411549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.1.47.189:39049)
0925 12:21:56.411692 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.411694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.411701 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:56.411718 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.411719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.411728 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.411787 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.412117 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.412826 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.413451 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.413661 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.413712 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.415235 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.415236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.415236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.415242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.417069 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.417462 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.417463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.417463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.417511 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.417703 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.417704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.417704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.417867 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.417953 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.418138 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.418275 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260925 12:21:56.421777 7844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Bootstrap starting.
I20260925 12:21:56.422636 7844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:56.422940 7844 log.cc:824] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:56.423930 7844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: No bootstrap required, opened a new log
I20260925 12:21:56.426000 7844 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } }
I20260925 12:21:56.426169 7844 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:56.426195 7844 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ee8a1711044aea950df8a6599c351d, State: Initialized, Role: FOLLOWER
I20260925 12:21:56.426317 7844 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } }
I20260925 12:21:56.426590 7844 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:56.426582 7851 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } } }
I20260925 12:21:56.426692 7851 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:21:56.428280 7862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:56.428345 7862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:56.430009 7776 master_runner.cc:428] Detected that this master a5ee8a1711044aea950df8a6599c351d is joining an existing cluster
I20260925 12:21:56.430048 7776 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.47.189:39049
I20260925 12:21:56.437922 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.431487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.1.47.190:37841)
0925 12:21:56.431537 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.431539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.431759 (+ 220us) server_negotiation.cc:407] Connection header received
0925 12:21:56.431929 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.431930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.431941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.431975 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.432312 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.432795 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.433558 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.433763 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.433848 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.434992 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.435001 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.435001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.435009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.436574 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437134 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.437135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.437136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.437198 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437401 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.437402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.437402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.437524 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.437595 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.437750 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.437826 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 12:21:56.437935 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.431580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44146)
0925 12:21:56.431703 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.431712 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.431793 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.432041 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.432041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.432195 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.432235 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.432236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.432877 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.432878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.433450 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.433451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.433867 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.433868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.433927 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.434408 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.434413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.434892 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.436676 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.436677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.436677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.436948 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.437274 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.437275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.437338 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.437597 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.437598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.437642 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.437722 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.437826 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260925 12:21:56.438066 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.431298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.1.47.189:39049)
0925 12:21:56.431344 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.431346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.431504 (+ 158us) server_negotiation.cc:407] Connection header received
0925 12:21:56.431555 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.431556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.431566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.431628 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.432092 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.432853 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.433906 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.434205 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.434245 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.435270 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.435271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.435271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.435277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.437006 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437374 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.437375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.437376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.437421 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437590 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.437591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.437591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.437703 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.437741 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.437908 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.437982 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:56.438685 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.431134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38622)
0925 12:21:56.431419 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.431430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.431494 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.431636 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.431637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.431918 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.432012 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.432014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.432983 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.432984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.433813 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.433815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.434312 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.434313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.434370 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.434751 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.434756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.435180 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.437091 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.437092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.437298 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.437482 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.437482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.437483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.437529 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.437781 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.437782 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.437815 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.438488 (+ 673us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.438590 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.190@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:56.447593 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.440234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38624)
0925 12:21:56.440374 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.440415 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.440489 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.440639 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.440641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.440881 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.440930 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.440931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.441820 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.441821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.442403 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.442405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.442842 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.442843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.442896 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.443371 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.443376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.444941 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.446573 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.446579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.446581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.446777 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.446970 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.446971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.446971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.447025 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.447278 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.447283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.447324 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.447388 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.447479 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":94,"threads_started":1}
I20260925 12:21:56.447602 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.440278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.1.47.189:39049)
0925 12:21:56.440316 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.440318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.440477 (+ 159us) server_negotiation.cc:407] Connection header received
0925 12:21:56.440564 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.440565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.440575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.440633 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.440996 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.441736 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.442475 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.442771 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.442843 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.445032 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.445034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.445034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.445042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.446491 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.446846 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.446847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.446847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.446906 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.447079 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.447080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.447080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.447217 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.447283 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.447431 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.447511 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:21:56.448261 7312 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.47.189:39049: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a5ee8a1711044aea950df8a6599c351d"
member_type: NON_VOTER
last_known_addr {
host: "127.1.47.189"
port: 39049
}
attrs {
promote: true
}
}
dest_uuid: "273fd950106a4234ad34dea2981ba481"
cas_config_opid_index: -1
I20260925 12:21:56.448594 7312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } }
I20260925 12:21:56.449121 7872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a5ee8a1711044aea950df8a6599c351d (127.1.47.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } }
I20260925 12:21:56.449383 7818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:56.450127 7818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 12:21:56.450162 7872 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.47.189:39049
I20260925 12:21:56.450239 7872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } }
I20260925 12:21:56.450299 7872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:56.450459 7776 master.cc:537] Master@127.1.47.189:39049 shutting down...
I20260925 12:21:56.450495 7871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 12:21:56.451315 7296 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 12:21:56.451699 7296 consensus_peers.cc:487] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 -> Peer a5ee8a1711044aea950df8a6599c351d (127.1.47.189:39049): Couldn't send request to peer a5ee8a1711044aea950df8a6599c351d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 12:21:56.452869 7776 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:56.452958 7776 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:56.453037 7776 tablet_replica.cc:330] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: stopping tablet replica
I20260925 12:21:56.465560 7776 master.cc:552] Master@127.1.47.189:39049 shutdown complete.
I20260925 12:21:56.465613 7776 master_runner.cc:305] Clearing existing system tablet
I20260925 12:21:56.465764 7776 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.465816 7776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.466446 7776 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260925 12:21:56.466938 7876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.467058 7776 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.467116 7776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.467137 7776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "a5ee8a1711044aea950df8a6599c351d"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "2b474d629edea60b98ae77ac95c3c40e"
server_key_iv: "99f7fbf5e681087d0a3c1603e4490c49"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.467775 7776 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:56.469389 7776 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 12:21:56.469460 7776 log.cc:1196] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 12:21:56.469722 7776 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Deleting consensus metadata
I20260925 12:21:56.469863 7776 master_runner.cc:315] Copying system tablet from 127.1.47.190:37841
I20260925 12:21:56.470572 7776 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: tablet copy: Beginning tablet copy session from remote peer at address 127.1.47.190:37841
I20260925 12:21:56.477490 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.470944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44160)
0925 12:21:56.471054 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.471066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.471140 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.471292 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.471293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.471458 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.471740 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.471742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.472784 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.472785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.473359 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.473362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.473839 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.473840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.473899 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.474421 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.474427 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.474739 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.476558 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.476559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.476559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.476726 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.476934 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.476935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.476935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.476984 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.477249 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.477249 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.477286 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.477343 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.477419 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":69,"threads_started":1}
I20260925 12:21:56.477559 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.471012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.1.47.190:37841)
0925 12:21:56.471069 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.471071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.471134 (+ 63us) server_negotiation.cc:407] Connection header received
0925 12:21:56.471193 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.471195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.471206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.471288 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.471815 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.472690 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.473447 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.473747 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.473820 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.474823 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.474824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.474825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.474832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.476476 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.476801 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.476803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.476803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.476866 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.477042 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.477042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.477043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.477184 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.477246 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.477389 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.477467 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 12:21:56.477815 7332 tablet_copy_service.cc:140] P 273fd950106a4234ad34dea2981ba481: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a5ee8a1711044aea950df8a6599c351d ({username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:44160)
I20260925 12:21:56.477928 7332 tablet_copy_service.cc:161] P 273fd950106a4234ad34dea2981ba481: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a5ee8a1711044aea950df8a6599c351d at {username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:44160: session id = a5ee8a1711044aea950df8a6599c351d-00000000000000000000000000000000
I20260925 12:21:56.478559 7332 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 12:21:56.479203 7776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 12:21:56.480548 7776 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: tablet copy: Starting download of 0 data blocks...
I20260925 12:21:56.480703 7776 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: tablet copy: Starting download of 1 WAL segments...
I20260925 12:21:56.481637 7776 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 12:21:56.482625 7332 tablet_copy_service.cc:342] P 273fd950106a4234ad34dea2981ba481: Request end of tablet copy session a5ee8a1711044aea950df8a6599c351d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:44160
I20260925 12:21:56.482690 7332 tablet_copy_service.cc:433] P 273fd950106a4234ad34dea2981ba481: ending tablet copy session a5ee8a1711044aea950df8a6599c351d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a5ee8a1711044aea950df8a6599c351d
I20260925 12:21:56.485376 7776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.486498 7884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.486650 7883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.486932 7886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.487156 7776 server_base.cc:1034] running on GCE node
I20260925 12:21:56.487250 7776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.487422 7776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.488546 7776 hybrid_clock.cc:648] HybridClock initialized: now 1790338916488536 us; error 39 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.489908 7776 init.cc:377] Logged in from keytab as oryx/127.1.47.189@KRBTEST.COM (short username oryx)
I20260925 12:21:56.490191 7776 webserver.cc:460] Webserver started at http://127.1.47.189:34427/ using document root <none> and password file <none>
I20260925 12:21:56.490310 7776 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.490360 7776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.490986 7776 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:56.491494 7893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.491616 7776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:56.491660 7776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "a5ee8a1711044aea950df8a6599c351d"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "2b474d629edea60b98ae77ac95c3c40e"
server_key_iv: "99f7fbf5e681087d0a3c1603e4490c49"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.491715 7776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.525439 7776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.527144 7776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.530375 7776 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.189:39049
I20260925 12:21:56.530632 7945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.189:39049 every 8 connection(s)
I20260925 12:21:56.530823 7776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
I20260925 12:21:56.531579 7946 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:21:56.532127 7946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Bootstrap starting.
I20260925 12:21:56.534379 7946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:21:56.534603 7946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Bootstrap complete.
I20260925 12:21:56.534803 7946 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } }
I20260925 12:21:56.534888 7946 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a5ee8a1711044aea950df8a6599c351d, State: Initialized, Role: LEARNER
I20260925 12:21:56.534938 7946 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } }
I20260925 12:21:56.535112 7946 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:56.535104 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } }
I20260925 12:21:56.535187 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: LEARNER
I20260925 12:21:56.536537 7960 catalog_manager.cc:1259] Loaded cluster ID: 1869eaf3bc064663b5fcc677050f4381
I20260925 12:21:56.536628 7960 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: loading cluster ID for follower catalog manager: success
I20260925 12:21:56.537830 7960 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: acquiring CA information for follower catalog manager: success
I20260925 12:21:56.538161 7960 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:21:56.571870 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.564622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.1.47.189:39049)
0925 12:21:56.564777 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.564780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.564787 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:56.564813 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.564814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.564825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.565009 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.565630 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.566340 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.567225 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.567452 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.567542 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.568925 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.568926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.568927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.568935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.570511 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.570909 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.570911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.570911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.570981 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.571206 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.571207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.571207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.571337 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.571503 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.571590 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.571703 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.1.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.188.keytab.
Entry for principal oryx/127.1.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.188.keytab.
I20260925 12:21:56.598884 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:36409
--webserver_interface=127.1.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.188.keytab
--principal=oryx/127.1.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:56.683262 7964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:56.683473 7964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:56.683494 7964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:56.684803 7964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:21:56.684850 7964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:56.684864 7964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:56.684877 7964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:21:56.684890 7964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:21:56.686332 7964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.188.keytab
--principal=oryx/127.1.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:36409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:56.686532 7964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:56.686767 7964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.691425 7970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.691425 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.691466 7972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.691975 7964 server_base.cc:1034] running on GCE node
I20260925 12:21:56.692214 7964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.692514 7964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.693697 7964 hybrid_clock.cc:648] HybridClock initialized: now 1790338916693678 us; error 30 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.696298 7964 init.cc:377] Logged in from keytab as oryx/127.1.47.188@KRBTEST.COM (short username oryx)
I20260925 12:21:56.696769 7964 webserver.cc:460] Webserver started at http://127.1.47.188:33561/ using document root <none> and password file <none>
I20260925 12:21:56.697002 7964 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.697069 7964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.697214 7964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:21:56.698361 7964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/instance:
uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "5c593e37a300e2719213df571d4dd78f"
server_key_iv: "0190bd2ab1e8b5a1d9463d623543cc3e"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.698747 7964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal/instance:
uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "5c593e37a300e2719213df571d4dd78f"
server_key_iv: "0190bd2ab1e8b5a1d9463d623543cc3e"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.700227 7964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:21:56.701017 7979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.701215 7964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:56.701313 7964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "5c593e37a300e2719213df571d4dd78f"
server_key_iv: "0190bd2ab1e8b5a1d9463d623543cc3e"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.701396 7964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.720674 7964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.722640 7964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:56.722818 7964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.727084 7964 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.188:36409
I20260925 12:21:56.727139 8031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.188:36409 every 8 connection(s)
I20260925 12:21:56.727499 7964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
I20260925 12:21:56.728866 8032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:56.730325 8032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:21:56.734004 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 7964
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.188@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
I20260925 12:21:56.740437 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.731202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.1.47.190:37841)
0925 12:21:56.731283 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.731285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.731432 (+ 147us) server_negotiation.cc:407] Connection header received
0925 12:21:56.732090 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.732091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.732104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.732153 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.732777 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.733279 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.734353 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.734646 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.734710 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.737228 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.737229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.737230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.737236 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.738908 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.739320 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.739323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.739323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.739382 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.739616 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.739617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.739617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.739766 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.739840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.740244 (+ 404us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.740330 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 12:21:56.740526 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.731089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44164)
0925 12:21:56.731310 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.731352 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.731982 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.732222 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.732226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.732493 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.732685 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.732689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.733375 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.733376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.734254 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.734256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.734749 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.734750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.734872 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.735443 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.735452 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.737108 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.739010 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.739017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.739019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.739243 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.739441 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.739442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.739442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.739542 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.739909 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.739916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.740074 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.740198 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.740321 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260925 12:21:56.741029 8032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:21:56.743644 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.734564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.1.47.188:36409)
0925 12:21:56.734750 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.734755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.734781 (+ 26us) server_negotiation.cc:407] Connection header received
0925 12:21:56.734803 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.734805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.734822 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.734924 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.735724 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.736729 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.737532 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.737999 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.738091 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.740722 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.740727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.740727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.740733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.742419 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.742814 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.742816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.742817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.742889 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.743072 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.743073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.743073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.743236 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.743395 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.743436 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.743520 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":128,"threads_started":1}
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.188@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:56.747838 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.741310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38670)
0925 12:21:56.741445 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.741456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.741532 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.741789 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.741791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.741955 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.742000 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.742002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.742684 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.742684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.743449 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.743451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.744213 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.744214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.744278 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.744602 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.744606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.745700 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.746969 (+ 1269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.746970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.746970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.747127 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.747347 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.747348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.747348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.747394 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.747635 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.747635 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.747668 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.747713 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.747777 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260925 12:21:56.747956 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.741390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.1.47.189:39049)
0925 12:21:56.741439 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.741440 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.741679 (+ 239us) server_negotiation.cc:407] Connection header received
0925 12:21:56.741702 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.741702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.741715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.741759 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.742074 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.742605 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.743542 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.743855 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.743937 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.745775 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.745776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.745777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.745782 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.746898 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.747214 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.747215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.747216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.747278 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.747461 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.747462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.747462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.747577 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.747636 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.747740 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.747812 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 12:21:56.748342 8032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.188@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
I20260925 12:21:56.754007 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.748600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:49604)
0925 12:21:56.748649 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.748660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.748696 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.748809 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.748810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.748912 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.748940 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.748941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.749558 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.749559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.750011 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.750012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.750375 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.750376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.750415 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.750819 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.750823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.752000 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.753234 (+ 1234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.753235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.753235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.753398 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.753550 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.753550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.753550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.753600 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.753773 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.753773 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.753806 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.753855 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.753929 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:21:56.754053 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.748652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.1.47.188:36409)
0925 12:21:56.748711 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.748713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.748720 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:56.748737 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.748737 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.748747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.748805 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.748993 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.749490 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.750086 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.750303 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.750360 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.752071 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.752072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.752072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.752078 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.753162 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.753455 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.753456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.753456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.753495 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.753638 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.753639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.753639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.753725 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.753768 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.753871 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.753970 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":42}
I20260925 12:21:56.755431 8032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Bootstrap starting.
I20260925 12:21:56.756381 8032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:56.756675 8032 log.cc:824] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:56.757438 8032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: No bootstrap required, opened a new log
I20260925 12:21:56.758865 8032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } }
I20260925 12:21:56.758994 8032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:56.759017 8032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e50c0ab9ca94adfa2aebf02947c1df1, State: Initialized, Role: FOLLOWER
I20260925 12:21:56.759169 8032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } }
I20260925 12:21:56.759508 8032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:56.759516 8037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } } }
I20260925 12:21:56.759692 8037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:21:56.761839 8048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:21:56.761919 8048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:21:56.764315 7964 master_runner.cc:428] Detected that this master 7e50c0ab9ca94adfa2aebf02947c1df1 is joining an existing cluster
I20260925 12:21:56.764375 7964 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.47.188:36409
I20260925 12:21:56.772739 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.766106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44170)
0925 12:21:56.766365 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.766384 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.766448 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.766620 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.766621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.766830 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.766880 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.766882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767778 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.767779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.768606 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.768607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.769084 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.769084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.769139 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.769651 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.769656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.769989 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.771740 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.771741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.771741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.771979 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.772173 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.772173 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.772174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.772225 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.772484 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.772485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.772536 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.772588 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.772662 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":71,"threads_started":1}
I20260925 12:21:56.772801 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.766144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.1.47.190:37841)
0925 12:21:56.766194 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.766196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.766426 (+ 230us) server_negotiation.cc:407] Connection header received
0925 12:21:56.766519 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.766520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.766531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.766561 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.766960 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767698 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.768689 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.768980 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.769015 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.770083 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.770085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.770085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.770093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.771653 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.772036 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.772038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.772038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.772106 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.772286 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.772287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.772287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.772422 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.772491 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.772634 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.772709 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338915, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.190@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
I20260925 12:21:56.791941 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.774744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:49616)
0925 12:21:56.774798 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.774808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.774845 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.775063 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.775064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.775211 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.775245 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.775247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.775950 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.775951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.776495 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.776497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.776881 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.776882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.776945 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.777359 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.777363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.778835 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.790564 (+ 11729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.790567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.790567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.790858 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.791104 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.791106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.791171 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.791508 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.791509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.791569 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.791624 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.791795 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:21:56.792472 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.774820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.1.47.188:36409)
0925 12:21:56.774941 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.774943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.774948 (+ 5us) server_negotiation.cc:407] Connection header received
0925 12:21:56.774976 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.774976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.774984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.775060 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.775328 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.775822 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.776590 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.776797 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.776882 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.778924 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.778926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.778926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.778933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.790444 (+ 11511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.790960 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.790962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.790962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.791021 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791247 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.791248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.791248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.791415 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.792198 (+ 783us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.792240 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.792329 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260925 12:21:56.793021 7312 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.47.188:36409: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
member_type: NON_VOTER
last_known_addr {
host: "127.1.47.188"
port: 36409
}
attrs {
promote: true
}
}
dest_uuid: "273fd950106a4234ad34dea2981ba481"
cas_config_opid_index: 5
I20260925 12:21:56.793118 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.765969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.1.47.188:36409)
0925 12:21:56.766006 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.766008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.766270 (+ 262us) server_negotiation.cc:407] Connection header received
0925 12:21:56.766346 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.766347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.766356 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.766418 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.766840 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767573 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.768671 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.768950 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.768989 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.789478 (+ 20489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.789480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.789480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.789487 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.791197 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791567 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.791568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.791569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.791620 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791811 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.791811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.791812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.791973 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.792023 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.792950 (+ 927us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.793037 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:56.793239 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.765750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.1.47.189:39049)
0925 12:21:56.765798 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.765800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.765922 (+ 122us) server_negotiation.cc:407] Connection header received
0925 12:21:56.766017 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.766018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.766028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.766058 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.766588 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767362 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.768681 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.768974 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.769018 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.790053 (+ 21035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.790055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.790055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.790064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.791978 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.792370 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.792372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.792372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.792425 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.792649 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.792650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.792651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.792794 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.792837 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.793047 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.793143 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:21:56.793308 7312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } }
I20260925 12:21:56.793763 7871 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.793879 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.765924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:49606)
0925 12:21:56.766223 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.766232 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.766282 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.766461 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.766462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.766697 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.766762 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.766764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767653 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.767653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.768579 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.768581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.769076 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.769077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.769134 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.788640 (+ 19506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.788648 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.789281 (+ 633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.791280 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791280 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.791281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.791491 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.791687 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.791687 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.791687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.791743 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.792401 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.792402 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.792446 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.793665 (+ 1219us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.793780 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":61,"threads_started":1}
I20260925 12:21:56.794199 8006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:56.794903 7871 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.47.188:36409
I20260925 12:21:56.794991 7871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.795066 7871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:56.795120 8006 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 12:21:56.795356 7964 master.cc:537] Master@127.1.47.188:36409 shutting down...
I20260925 12:21:56.795622 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7374
I20260925 12:21:56.798672 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.765658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38678)
0925 12:21:56.765859 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.765870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.765945 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.766201 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.766201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.766414 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.766496 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.766498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.767586 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.767587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.768578 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.768580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.769091 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.769092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.769170 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.788869 (+ 19699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.788875 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.789701 (+ 826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.792073 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.792073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.792074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.792287 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.792492 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.792492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.792493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.792544 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.792899 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.792900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.792938 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.793350 (+ 412us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.793457 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":31,"thread_start_us":97,"threads_started":1}
I20260925 12:21:56.798871 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.793773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:38686)
0925 12:21:56.795287 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.795299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.795347 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.795576 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.795577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.795578 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:56.795620 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.795621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.796479 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.796480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.797680 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.797683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.798524 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.798525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.798615 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.798643 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.798707 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.798760 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495}
I20260925 12:21:56.799353 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.793857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38686 (local address 127.1.47.189:39049)
0925 12:21:56.795417 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.795419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.795425 (+ 6us) server_negotiation.cc:407] Connection header received
0925 12:21:56.795445 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.795446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.795467 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.795523 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:56.795686 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.796385 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.797778 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.798437 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.798511 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.798588 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.799249 (+ 661us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.799290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1540}
I20260925 12:21:56.799546 7920 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 LEARNER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 12:21:56.802150 7872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 12:21:56.802420 7871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:56.802579 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:43517
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:37841
--webserver_port=34627
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:21:56.802841 7920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
W20260925 12:21:56.803414 7298 consensus_peers.cc:487] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 -> Peer 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409): Couldn't send request to peer 7e50c0ab9ca94adfa2aebf02947c1df1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 12:21:56.804584 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273fd950106a4234ad34dea2981ba481. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.804699 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: LEARNER
I20260925 12:21:56.804755 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: NON_VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: true } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.804814 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: LEARNER
I20260925 12:21:56.805042 7964 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:56.805106 7871 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: attempting to promote NON_VOTER a5ee8a1711044aea950df8a6599c351d to VOTER
I20260925 12:21:56.805166 7964 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:56.805217 7964 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: stopping tablet replica
I20260925 12:21:56.805317 7871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } }
W20260925 12:21:56.806164 7298 consensus_peers.cc:487] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 -> Peer 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409): Couldn't send request to peer 7e50c0ab9ca94adfa2aebf02947c1df1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 12:21:56.806452 7920 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 LEARNER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 12:21:56.806630 7871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 12:21:56.807288 7871 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a5ee8a1711044aea950df8a6599c351d (127.1.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.808329 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.808164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:49620)
0925 12:21:56.808248 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.808287 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:36409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 12:21:56.808413 7871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.808501 7871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:56.808562 7871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.808540 7920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a5ee8a1711044aea950df8a6599c351d (127.1.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.808619 7871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:56.809305 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.809418 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:21:56.818722 7964 master.cc:552] Master@127.1.47.188:36409 shutdown complete.
I20260925 12:21:56.818799 7964 master_runner.cc:305] Clearing existing system tablet
I20260925 12:21:56.818938 7964 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.819002 7964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.819653 7964 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:21:56.820196 8063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.820387 7964 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.820447 7964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.820477 7964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "5c593e37a300e2719213df571d4dd78f"
server_key_iv: "0190bd2ab1e8b5a1d9463d623543cc3e"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.821141 7964 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:56.822602 7964 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 12:21:56.822686 7964 log.cc:1196] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 12:21:56.822887 7964 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Deleting consensus metadata
I20260925 12:21:56.823061 7964 master_runner.cc:315] Copying system tablet from 127.1.47.190:37841
I20260925 12:21:56.823685 7964 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: tablet copy: Beginning tablet copy session from remote peer at address 127.1.47.190:37841
I20260925 12:21:56.830276 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.824251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:44174)
0925 12:21:56.824377 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.824387 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.824470 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.824597 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.824598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.824781 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.824823 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.824825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.825498 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.825499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.826074 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.826077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.826459 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.826460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.826516 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.827031 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.827037 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.827349 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.829169 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.829170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.829170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.829351 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.829572 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.829573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.829573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.829632 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.829923 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.829924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.829970 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.830059 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.830170 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260925 12:21:56.830265 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.824284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.1.47.190:37841)
0925 12:21:56.824332 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.824335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.824455 (+ 120us) server_negotiation.cc:407] Connection header received
0925 12:21:56.824537 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.824538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.824548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.824595 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.824904 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.825412 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.826165 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.826375 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.826448 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.827457 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.827458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.827459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.827466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.829064 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.829439 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.829440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.829441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.829495 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.829731 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.829732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.829733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.829857 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.829932 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.830089 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.830169 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:21:56.830937 7332 tablet_copy_service.cc:140] P 273fd950106a4234ad34dea2981ba481: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e50c0ab9ca94adfa2aebf02947c1df1 ({username='oryx', principal='oryx/127.1.47.188@KRBTEST.COM'} at 127.0.0.1:44174)
I20260925 12:21:56.831022 7332 tablet_copy_service.cc:161] P 273fd950106a4234ad34dea2981ba481: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e50c0ab9ca94adfa2aebf02947c1df1 at {username='oryx', principal='oryx/127.1.47.188@KRBTEST.COM'} at 127.0.0.1:44174: session id = 7e50c0ab9ca94adfa2aebf02947c1df1-00000000000000000000000000000000
I20260925 12:21:56.831480 7332 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 12:21:56.832088 7964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 12:21:56.833110 7964 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: tablet copy: Starting download of 0 data blocks...
I20260925 12:21:56.833272 7964 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: tablet copy: Starting download of 1 WAL segments...
I20260925 12:21:56.833992 7964 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 12:21:56.834724 7332 tablet_copy_service.cc:342] P 273fd950106a4234ad34dea2981ba481: Request end of tablet copy session 7e50c0ab9ca94adfa2aebf02947c1df1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.47.188@KRBTEST.COM'} at 127.0.0.1:44174
I20260925 12:21:56.834779 7332 tablet_copy_service.cc:433] P 273fd950106a4234ad34dea2981ba481: ending tablet copy session 7e50c0ab9ca94adfa2aebf02947c1df1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e50c0ab9ca94adfa2aebf02947c1df1
I20260925 12:21:56.837177 7964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.838482 8073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.838557 8070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.838866 8071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.844117 7964 server_base.cc:1034] running on GCE node
I20260925 12:21:56.844237 7964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.844455 7964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.845610 7964 hybrid_clock.cc:648] HybridClock initialized: now 1790338916845601 us; error 31 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.847182 7964 init.cc:377] Logged in from keytab as oryx/127.1.47.188@KRBTEST.COM (short username oryx)
I20260925 12:21:56.847502 7964 webserver.cc:460] Webserver started at http://127.1.47.188:35095/ using document root <none> and password file <none>
I20260925 12:21:56.847640 7964 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.847681 7964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.848496 7964 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.849084 8080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.849251 7964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:21:56.849303 7964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "5c593e37a300e2719213df571d4dd78f"
server_key_iv: "0190bd2ab1e8b5a1d9463d623543cc3e"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.849371 7964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.867436 7964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.869395 7964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.874221 7964 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.188:36409
I20260925 12:21:56.874236 8132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.188:36409 every 8 connection(s)
I20260925 12:21:56.874598 7964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
I20260925 12:21:56.875757 8133 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:21:56.876343 8133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Bootstrap starting.
I20260925 12:21:56.878849 8133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:21:56.879093 8133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: Bootstrap complete.
I20260925 12:21:56.879298 8133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } }
I20260925 12:21:56.879400 8133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e50c0ab9ca94adfa2aebf02947c1df1, State: Initialized, Role: LEARNER
I20260925 12:21:56.879447 8133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } }
I20260925 12:21:56.879664 8133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:21:56.879621 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:56.879823 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: LEARNER
I20260925 12:21:56.881532 8147 catalog_manager.cc:1259] Loaded cluster ID: 1869eaf3bc064663b5fcc677050f4381
I20260925 12:21:56.881637 8147 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: loading cluster ID for follower catalog manager: success
I20260925 12:21:56.882845 8147 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: acquiring CA information for follower catalog manager: success
I20260925 12:21:56.883177 8147 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 12:21:56.928005 8059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:56.928236 8059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:56.928262 8059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:56.929685 8059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:56.929733 8059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:56.929803 8059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:21:56.931629 8059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:43517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=34627
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:56.931933 8059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:56.932277 8059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:56.938086 8155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.938253 8152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:56.938436 8153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:56.938799 8059 server_base.cc:1034] running on GCE node
I20260925 12:21:56.939119 8059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:56.939384 8059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:56.940549 8059 hybrid_clock.cc:648] HybridClock initialized: now 1790338916940526 us; error 39 us; skew 500 ppm
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:56.943328 8059 init.cc:377] Logged in from keytab as oryx/127.1.47.129@KRBTEST.COM (short username oryx)
I20260925 12:21:56.943949 8059 webserver.cc:460] Webserver started at http://127.1.47.129:34627/ using document root <none> and password file <none>
I20260925 12:21:56.944216 8059 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:56.944304 8059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:56.945941 8059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:21:56.946664 8162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.946857 8059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:56.946962 8059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "a584399904fb4e319b34991da3a8317f"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "d865b4e0e64a60ff7ecff0ed63f521cf"
server_key_iv: "7a2512a2f18be68cc25695545fea9402"
server_key_version: "encryptionkey@0"
I20260925 12:21:56.947365 8059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:56.970868 8059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:56.973742 8059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:56.973933 8059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:56.974215 8059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:56.974570 8059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:56.974627 8059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.974673 8059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:56.974722 8059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:56.982290 8059 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43517
I20260925 12:21:56.982331 8275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43517 every 8 connection(s)
I20260925 12:21:56.982810 8059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
I20260925 12:21:56.990535 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8059
I20260925 12:21:56.990694 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7508
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
Sep 25 12:21:56 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338916, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:56.995396 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:49857 (local address 127.1.47.188:36409)
0925 12:21:56.984082 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.984084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.984241 (+ 157us) server_negotiation.cc:407] Connection header received
0925 12:21:56.984990 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.984991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.985005 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.985537 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.986006 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.987097 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988011 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988294 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988373 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.991128 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.991129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.991130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.991136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.993193 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.993803 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.993804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.993804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.993867 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.994190 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.994191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.994191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.994374 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.994439 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.995184 (+ 745us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.995289 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":67,"threads_started":1}
I20260925 12:21:56.995533 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:32875 (local address 127.1.47.190:37841)
0925 12:21:56.983818 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.983821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.984369 (+ 548us) server_negotiation.cc:407] Connection header received
0925 12:21:56.984893 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.984894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.984908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.984942 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.985983 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.986834 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.987781 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988109 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988168 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.992358 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.992359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.992360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.992368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.994225 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.994625 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.994627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.994627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.994690 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.994903 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.994904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.994904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.995049 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.995096 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.995349 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.995432 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:21:56.995922 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.1.47.129:49857)
0925 12:21:56.984079 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.984131 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.984729 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.985593 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.985594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.985820 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.985902 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.985904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.987184 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.987186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.987930 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.987932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988372 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.988453 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.989178 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.989183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.991017 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.993287 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.993295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.993299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.993705 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.993944 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.993945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.993945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.994092 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.994480 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.994488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.994683 (+ 195us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.994810 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.995008 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":10,"thread_start_us":89,"threads_started":1}
I20260925 12:21:56.996156 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:32875)
0925 12:21:56.984322 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.984330 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.984743 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.985048 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.985053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.985767 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.985861 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.985869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.986934 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.986935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.987694 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.987696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988163 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.988336 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.989129 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.989141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.992199 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.994310 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.994310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.994311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.994538 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.994769 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.994770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.994770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.994829 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.995150 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.995152 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.995192 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.995912 (+ 720us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.996021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260925 12:21:56.996410 8276 heartbeater.cc:347] Connected to a master server at 127.1.47.188:36409
I20260925 12:21:56.996511 8276 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:56.996691 8277 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:56.996773 8277 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:56.996809 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33145 (local address 127.1.47.189:39049)
0925 12:21:56.983900 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:56.983902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:56.984241 (+ 339us) server_negotiation.cc:407] Connection header received
0925 12:21:56.984839 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:56.984841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:56.984856 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:56.984920 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:56.985962 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.986444 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.987631 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988024 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988092 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.993027 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:56.993029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:56.993029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:56.993037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:56.995017 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.995453 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.995454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.995454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.995508 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:56.996174 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:56.996176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:56.996176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:56.996347 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:56.996390 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:56.996606 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 12:21:56.996689 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:21:56.996855 8277 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:56.997376 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:56.983714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.1.47.129:33145)
0925 12:21:56.984141 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:56.984149 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:56.984729 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:56.985276 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:56.985277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:56.985659 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:56.985861 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.985866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.986549 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.986549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.987535 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:56.987537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:56.988161 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:56.988162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:56.988336 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:56.989129 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:56.989141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:56.992922 (+ 3781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:56.995115 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.995115 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.995116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.995374 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.995577 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:56.995577 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:56.995578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:56.995645 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:56.996440 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:56.996441 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:56.996481 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 12:21:56.996927 (+ 446us) client_negotiation.cc:240] Negotiation successful
0925 12:21:56.997043 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260925 12:21:56.997573 7312 ts_manager.cc:194] Re-registered known tserver with Master: a584399904fb4e319b34991da3a8317f (127.1.47.129:43517)
I20260925 12:21:56.997753 8278 heartbeater.cc:347] Connected to a master server at 127.1.47.189:39049
I20260925 12:21:56.997807 8278 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:56.997879 8278 heartbeater.cc:511] Master 127.1.47.189:39049 requested a full tablet report, sending...
I20260925 12:21:56.998129 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:32875
I20260925 12:21:56.998286 7910 ts_manager.cc:194] Registered new tserver with Master: a584399904fb4e319b34991da3a8317f (127.1.47.129:43517)
I20260925 12:21:56.996713 8276 heartbeater.cc:511] Master 127.1.47.188:36409 requested a full tablet report, sending...
I20260925 12:21:57.000329 8089 ts_manager.cc:194] Registered new tserver with Master: a584399904fb4e319b34991da3a8317f (127.1.47.129:43517)
I20260925 12:21:57.001368 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:38601
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:37841
--webserver_port=42903
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:21:57.091223 8284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:57.091507 8284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:57.091548 8284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:57.092905 8284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:57.092984 8284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:57.093030 8284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:21:57.094709 8284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:38601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=42903
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:57.095003 8284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:57.095265 8284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:57.100703 8289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:57.100708 8290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:57.100683 8292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:57.100937 8284 server_base.cc:1034] running on GCE node
I20260925 12:21:57.101141 8284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:57.101403 8284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:57.102543 8284 hybrid_clock.cc:648] HybridClock initialized: now 1790338917102532 us; error 27 us; skew 500 ppm
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:57.105464 8284 init.cc:377] Logged in from keytab as oryx/127.1.47.130@KRBTEST.COM (short username oryx)
I20260925 12:21:57.105991 8284 webserver.cc:460] Webserver started at http://127.1.47.130:42903/ using document root <none> and password file <none>
I20260925 12:21:57.106230 8284 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:57.106314 8284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:57.107805 8284 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:21:57.108460 8299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.108644 8284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.108757 8284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "872ec0ce067c43e087553da603ca6e02"
format_stamp: "Formatted at 2026-09-25 12:21:55 on dist-test-slave-l9ww"
server_key: "c19047c0512da3e9d8fdd7d0bf5485d9"
server_key_iv: "535ef35a98e9df4f1833ee6376b73702"
server_key_version: "encryptionkey@0"
I20260925 12:21:57.109129 8284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:57.127681 8284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:57.130330 8284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:57.130631 8284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:57.130890 8284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:57.131217 8284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:57.131270 8284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.131312 8284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:57.131359 8284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.137915 8284 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:38601
I20260925 12:21:57.138024 8412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:38601 every 8 connection(s)
I20260925 12:21:57.138405 8284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:57.146636 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8284
I20260925 12:21:57.146806 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7642
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:57.150554 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:49379 (local address 127.1.47.188:36409)
0925 12:21:57.139620 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.139622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.139631 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:57.140354 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.140356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.140369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.140401 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.141290 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.141804 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143299 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143639 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143711 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.146460 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.146462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.146462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.146472 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.148874 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.149407 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.149409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.149409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.149475 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.149766 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.149767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.149767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.149923 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.149994 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.150318 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.150424 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 12:21:57.150767 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:51175 (local address 127.1.47.189:39049)
0925 12:21:57.139472 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.139475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.139687 (+ 212us) server_negotiation.cc:407] Connection header received
0925 12:21:57.140325 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.140326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.140339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.140399 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.141928 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.142638 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143396 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143660 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143740 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.147730 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.147731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.147732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.147740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.149540 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.149966 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.149968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.149968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.150034 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.150226 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.150227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.150227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.150353 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.150412 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.150595 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.150681 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260925 12:21:57.150794 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.1.47.130:49379)
0925 12:21:57.139495 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.139536 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.140233 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.140474 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.140475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.140970 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.141186 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.141191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.141917 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.141918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.143199 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.143202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143732 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.143900 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.144663 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.144668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.146325 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.148964 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.148975 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.148978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.149317 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.149552 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.149553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.149553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.149708 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.149980 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.149986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.150187 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.150297 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.150496 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260925 12:21:57.151353 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.1.47.130:51175)
0925 12:21:57.139591 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.139601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.140225 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.140471 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.140474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.140971 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.141187 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.141193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.142732 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.142732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.143312 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.143314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143791 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.143899 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.144601 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.144612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.147636 (+ 3024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.149635 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.149636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.149637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.149860 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.150096 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.150097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.150097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.150156 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.150438 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.150439 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.150480 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.151116 (+ 636us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.151230 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":12,"thread_start_us":92,"threads_started":1}
I20260925 12:21:57.151379 8413 heartbeater.cc:347] Connected to a master server at 127.1.47.188:36409
I20260925 12:21:57.151470 8413 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.151674 8413 heartbeater.cc:511] Master 127.1.47.188:36409 requested a full tablet report, sending...
I20260925 12:21:57.152256 8089 ts_manager.cc:194] Registered new tserver with Master: 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:21:57.152524 8415 heartbeater.cc:347] Connected to a master server at 127.1.47.189:39049
I20260925 12:21:57.152580 8415 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.152652 8415 heartbeater.cc:511] Master 127.1.47.189:39049 requested a full tablet report, sending...
I20260925 12:21:57.153457 7910 ts_manager.cc:194] Registered new tserver with Master: 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:21:57.154274 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:45425
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:37841
--webserver_port=38031
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:21:57.155398 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:59155 (local address 127.1.47.190:37841)
0925 12:21:57.139491 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.139493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.140176 (+ 683us) server_negotiation.cc:407] Connection header received
0925 12:21:57.140454 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.140455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.140466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.140522 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.142749 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143470 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.144404 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.144726 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.144797 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.152525 (+ 7728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.152527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.152527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.152535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.154130 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.154555 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.154556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.154557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.154630 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.154839 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.154840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.154840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.154993 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.155055 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.155169 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.155259 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:21:57.155485 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.139195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:59155)
0925 12:21:57.140098 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.140108 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.140232 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.140554 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.140555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.140977 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.141188 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.141193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.143546 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.143547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.144303 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.144305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.144853 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.144854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.144927 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.145335 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.145341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.152439 (+ 7098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.154230 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.154232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.154232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.154440 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.154702 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.154702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.154703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.154757 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.155050 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.155051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.155087 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.155131 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.155210 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":46,"thread_start_us":76,"threads_started":1}
I20260925 12:21:57.155936 8414 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:57.155989 8414 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.156075 8414 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:57.156332 7312 ts_manager.cc:194] Re-registered known tserver with Master: 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:21:57.156721 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:59155
W20260925 12:21:57.240324 8420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:21:57.240623 8420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:21:57.240665 8420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:21:57.241997 8420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:21:57.242075 8420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:21:57.242122 8420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:21:57.243706 8420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:45425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=38031
--tserver_master_addrs=127.1.47.190:37841,127.1.47.189:39049,127.1.47.188:36409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:21:57.244048 8420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:21:57.244295 8420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:21:57.249459 8425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:57.249488 8426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:21:57.249537 8428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:21:57.249752 8420 server_base.cc:1034] running on GCE node
I20260925 12:21:57.250000 8420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:21:57.250281 8420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:21:57.251472 8420 hybrid_clock.cc:648] HybridClock initialized: now 1790338917251460 us; error 32 us; skew 500 ppm
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:21:57.254182 8420 init.cc:377] Logged in from keytab as oryx/127.1.47.131@KRBTEST.COM (short username oryx)
I20260925 12:21:57.254591 8420 webserver.cc:460] Webserver started at http://127.1.47.131:38031/ using document root <none> and password file <none>
I20260925 12:21:57.254837 8420 fs_manager.cc:359] Metadata directory not provided
I20260925 12:21:57.254922 8420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:21:57.256464 8420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:21:57.257169 8435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.257395 8420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:21:57.257509 8420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "53d3e915dee14c3c96ff541f6cd61fa2"
format_stamp: "Formatted at 2026-09-25 12:21:56 on dist-test-slave-l9ww"
server_key: "821070ab9eda24269f97af26cc1fbf4e"
server_key_iv: "b0eabf47201c92a00714541660950461"
server_key_version: "encryptionkey@0"
I20260925 12:21:57.257896 8420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:21:57.269513 8420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:21:57.272023 8420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:21:57.272192 8420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:21:57.272437 8420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:21:57.272763 8420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:21:57.272817 8420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.272862 8420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:21:57.272891 8420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:21:57.279191 8420 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:45425
I20260925 12:21:57.279242 8548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:45425 every 8 connection(s)
I20260925 12:21:57.279624 8420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:21:57.279891 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8420
I20260925 12:21:57.281780 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.278025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:49634)
0925 12:21:57.278081 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.278093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.278142 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.278322 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.278323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.278325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.278377 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.278380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.279355 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.279357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.280554 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.280559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.281503 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.281504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.281576 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.281580 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.281630 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.281683 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:21:57.282769 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.278067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.1.47.188:36409)
0925 12:21:57.278110 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.278112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.278125 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:21:57.278228 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.278229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.278254 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.278317 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.278469 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.279254 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.280648 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.281305 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.281374 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.281456 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.282657 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.282706 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:21:57.283521 8107 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260925 12:21:57.283730 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273fd950106a4234ad34dea2981ba481. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:57.283823 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: LEARNER
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.188@KRBTEST.COM
I20260925 12:21:57.289927 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.281262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.1.47.188:36409)
0925 12:21:57.281991 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.281993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.282154 (+ 161us) server_negotiation.cc:407] Connection header received
0925 12:21:57.282176 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.282177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.282189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.282257 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.282962 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.283529 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.284689 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.284980 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.285040 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.286596 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.286597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.286597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.286603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.288393 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.288841 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.288843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.288844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.288912 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.289135 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.289135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.289136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.289265 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.289364 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.289704 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.289791 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":47,"threads_started":1}
I20260925 12:21:57.290743 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.281044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.1.47.189:39049)
0925 12:21:57.281083 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.281085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.281232 (+ 147us) server_negotiation.cc:407] Connection header received
0925 12:21:57.281553 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.281554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.281565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.281609 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.282668 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.283390 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.285058 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.285351 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.285391 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.286846 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.286848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.286848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.286857 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.288479 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.289678 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.289679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.289679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.289742 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.289960 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.289961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.289961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.290105 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.290170 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.290553 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.290641 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:21:57.293035 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.281454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.1.47.190:37841)
0925 12:21:57.281496 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.281498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.283846 (+ 2348us) server_negotiation.cc:407] Connection header received
0925 12:21:57.283958 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.283960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.283971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.284001 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.284725 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.285402 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.286508 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.286789 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.286834 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.289371 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.289372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.289373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.289381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.291163 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.291645 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.291647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.291647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.291707 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.292434 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.292435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.292435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.292568 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.292644 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.292856 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.292930 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.189@KRBTEST.COM
I20260925 12:21:57.294713 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.280820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:36441 (local address 127.1.47.188:36409)
0925 12:21:57.280908 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.280910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.280918 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:57.281743 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.281744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.281758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.281801 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.283204 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.284064 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.286132 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.286476 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.286521 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.290765 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.290767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.290767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.290774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.292818 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.293389 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.293390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.293390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.293451 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.293748 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.293749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.293749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.293886 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.293929 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.294535 (+ 606us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.294624 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":44,"threads_started":1}
I20260925 12:21:57.295341 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.280550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.1.47.131:36441)
0925 12:21:57.280797 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.280845 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.281621 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.282008 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.282012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.282812 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.283103 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.283109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.284181 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.284182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.286020 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.286023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.286610 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.286611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.286757 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.288211 (+ 1454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.288221 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.290660 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.292899 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.292907 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.292910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.293226 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.293520 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.293521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.293521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.293675 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.293978 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.293985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.294355 (+ 370us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.294888 (+ 533us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.295084 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260925 12:21:57.295804 8549 heartbeater.cc:347] Connected to a master server at 127.1.47.188:36409
I20260925 12:21:57.295940 8549 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.296177 8549 heartbeater.cc:511] Master 127.1.47.188:36409 requested a full tablet report, sending...
I20260925 12:21:57.296638 8089 ts_manager.cc:194] Registered new tserver with Master: 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
I20260925 12:21:57.295296 7312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:21:57.297722 7312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 25 12:21:57 dist-test-slave-l9ww krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338917, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:21:57.297804 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.282149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:44223 (local address 127.1.47.189:39049)
0925 12:21:57.285743 (+ 3594us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.285745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.285753 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:57.285773 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.285774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.285787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.285848 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.287494 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.288276 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.291017 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.291436 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.291496 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.294010 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.294012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.294012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.294018 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.296086 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.296474 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.296475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.296475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.296541 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.296739 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.296740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.296741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.297217 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.297281 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.297583 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.297680 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3576,"thread_start_us":46,"threads_started":1}
I20260925 12:21:57.298154 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.281787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.1.47.131:44223)
0925 12:21:57.283663 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.283673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.283712 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.287106 (+ 3394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.287107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.287314 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.287407 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.287409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.288476 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.288477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.289273 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.289276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.291548 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.291549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.291637 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.292301 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.292306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.293925 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.296192 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.296193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.296194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.296394 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.296617 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.296618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.296618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.296673 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.297319 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.297320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.297365 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.297443 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.298060 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1858,"thread_start_us":69,"threads_started":1}
I20260925 12:21:57.298843 8551 heartbeater.cc:347] Connected to a master server at 127.1.47.189:39049
I20260925 12:21:57.298900 8551 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.298851 8061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: NON_VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: true } } }
I20260925 12:21:57.298980 8551 heartbeater.cc:511] Master 127.1.47.189:39049 requested a full tablet report, sending...
I20260925 12:21:57.298980 8061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:57.299350 7910 ts_manager.cc:194] Registered new tserver with Master: 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
I20260925 12:21:57.300433 8061 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: attempting to promote NON_VOTER 7e50c0ab9ca94adfa2aebf02947c1df1 to VOTER
I20260925 12:21:57.300577 8061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:21:57.300976 8107 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 LEARNER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 12:21:57.300989 7920 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Refusing update from remote peer 273fd950106a4234ad34dea2981ba481: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 12:21:57.301160 7872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 12:21:57.301263 7871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 12:21:57.301880 7872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.302039 7920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.302418 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.282395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:53941)
0925 12:21:57.290766 (+ 8371us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.290778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.290937 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.292027 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.292027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.292192 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:21:57.292297 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.292300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.293655 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.293656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.294478 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.294481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.295126 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.295127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.295195 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.296514 (+ 1319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:21:57.296519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:21:57.298206 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:21:57.300766 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.300767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.300767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.301036 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.301272 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:21:57.301272 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:21:57.301273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:21:57.301338 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:21:57.302089 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:21:57.302090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:21:57.302138 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.302189 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.302320 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8352,"thread_start_us":66,"threads_started":1}
I20260925 12:21:57.302855 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.282546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:53941 (local address 127.1.47.190:37841)
0925 12:21:57.291841 (+ 9295us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.291843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.291851 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:21:57.291940 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.291941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.291955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.292023 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.292423 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.293565 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.294573 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.295056 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.295127 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.298300 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.298301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.298302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.298308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.300653 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.301139 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.301140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.301141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.301211 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.301704 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.301706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.301706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.301978 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.302539 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.302660 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.302757 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9265,"thread_start_us":78,"threads_started":1}
I20260925 12:21:57.302976 7872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.303079 7872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:57.302968 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.303130 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:57.303164 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.303244 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:21:57.303565 8550 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37841
I20260925 12:21:57.303628 8550 heartbeater.cc:464] Registering TS with master...
I20260925 12:21:57.303702 8550 heartbeater.cc:511] Master 127.1.47.190:37841 requested a full tablet report, sending...
I20260925 12:21:57.304221 7872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.304311 7872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481 [sys.catalog]: This master's current role is: LEADER
I20260925 12:21:57.304435 7312 ts_manager.cc:194] Re-registered known tserver with Master: 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
I20260925 12:21:57.304502 8107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.304917 7312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:53941
I20260925 12:21:57.305806 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273fd950106a4234ad34dea2981ba481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:21:57.305995 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:21:57.310819 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.306986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43517 (local address 127.0.0.1:60954)
0925 12:21:57.307188 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.307200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.307245 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.307681 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.307682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.307684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.307734 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.307735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.308654 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.308655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.309667 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.309669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.310595 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.310596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.310650 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.310654 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.310698 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.310744 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260925 12:21:57.310832 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.307349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.1.47.129:43517)
0925 12:21:57.307518 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.307522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.307531 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:57.307566 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.307568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.307583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.307685 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.307790 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.308563 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.309767 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.310515 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.310598 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.310680 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.310709 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.310747 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260925 12:21:57.311370 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.307051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45425 (local address 127.0.0.1:41788)
0925 12:21:57.307415 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.307423 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.307459 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.307741 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.307742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.307745 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.307795 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.307797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.309144 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.309145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.310250 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.310253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.311118 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.311119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.311182 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.311187 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.311241 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.311280 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260925 12:21:57.311605 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.307006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:38601 (local address 127.0.0.1:51606)
0925 12:21:57.307285 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.307294 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.307337 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.307959 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.307960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.307962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.307998 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.307999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.309343 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.309344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.310065 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.310067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.311367 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.311368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.311430 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.311434 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.311500 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.311534 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":62,"threads_started":1}
I20260925 12:21:57.311602 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.307126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.1.47.131:45425)
0925 12:21:57.307568 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.307572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.307581 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:57.307620 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.307622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.307640 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.307921 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.307989 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.309052 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.310336 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.311037 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.311107 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.311183 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.311470 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.311523 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":91,"threads_started":1}
I20260925 12:21:57.311650 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.307582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.1.47.130:38601)
0925 12:21:57.307764 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.307767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.307778 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:21:57.307813 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.307816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.307831 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.308071 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.308124 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.309256 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.310163 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.311305 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.311370 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.311435 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.311523 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.311566 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260925 12:21:57.311944 8210 tablet_service.cc:1467] Processing CreateTablet for tablet 98c93d36042a4fda8cbb9287334284aa (DEFAULT_TABLE table=test-table [id=2a3600cfeeb74f938ed642df977b8ff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:57.312321 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98c93d36042a4fda8cbb9287334284aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:57.312460 8347 tablet_service.cc:1467] Processing CreateTablet for tablet 98c93d36042a4fda8cbb9287334284aa (DEFAULT_TABLE table=test-table [id=2a3600cfeeb74f938ed642df977b8ff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:57.312803 8347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98c93d36042a4fda8cbb9287334284aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:57.315910 8580 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Bootstrap starting.
I20260925 12:21:57.316694 8483 tablet_service.cc:1467] Processing CreateTablet for tablet 98c93d36042a4fda8cbb9287334284aa (DEFAULT_TABLE table=test-table [id=2a3600cfeeb74f938ed642df977b8ff8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:21:57.316870 8580 tablet_bootstrap.cc:654] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:57.317097 8483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98c93d36042a4fda8cbb9287334284aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:21:57.317210 8580 log.cc:824] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:57.317816 8581 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Bootstrap starting.
I20260925 12:21:57.317991 8580 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: No bootstrap required, opened a new log
I20260925 12:21:57.318061 8580 ts_tablet_manager.cc:1397] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:21:57.318485 8581 tablet_bootstrap.cc:654] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:57.318821 8581 log.cc:824] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:57.319621 8581 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: No bootstrap required, opened a new log
I20260925 12:21:57.319691 8581 ts_tablet_manager.cc:1397] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:21:57.319823 8580 raft_consensus.cc:348] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.320015 8580 raft_consensus.cc:374] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:57.320052 8580 raft_consensus.cc:729] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a584399904fb4e319b34991da3a8317f, State: Initialized, Role: FOLLOWER
I20260925 12:21:57.320188 8580 consensus_queue.cc:260] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.320878 8580 ts_tablet_manager.cc:1428] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 12:21:57.320860 8584 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Bootstrap starting.
I20260925 12:21:57.321012 8277 heartbeater.cc:503] Master 127.1.47.190:37841 was elected leader, sending a full tablet report...
I20260925 12:21:57.321686 8584 tablet_bootstrap.cc:654] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Neither blocks nor log segments found. Creating new log.
I20260925 12:21:57.322136 8584 log.cc:824] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Log is configured to *not* fsync() on all Append() calls
I20260925 12:21:57.322731 8581 raft_consensus.cc:348] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.322885 8581 raft_consensus.cc:374] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:57.322932 8581 raft_consensus.cc:729] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872ec0ce067c43e087553da603ca6e02, State: Initialized, Role: FOLLOWER
I20260925 12:21:57.323050 8581 consensus_queue.cc:260] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.323307 8581 ts_tablet_manager.cc:1428] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 12:21:57.323326 8414 heartbeater.cc:503] Master 127.1.47.190:37841 was elected leader, sending a full tablet report...
I20260925 12:21:57.323815 8584 tablet_bootstrap.cc:492] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: No bootstrap required, opened a new log
I20260925 12:21:57.323912 8584 ts_tablet_manager.cc:1397] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 12:21:57.325950 8584 raft_consensus.cc:348] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.326081 8584 raft_consensus.cc:374] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:21:57.326103 8584 raft_consensus.cc:729] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d3e915dee14c3c96ff541f6cd61fa2, State: Initialized, Role: FOLLOWER
I20260925 12:21:57.326227 8584 consensus_queue.cc:260] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.326501 8584 ts_tablet_manager.cc:1428] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:21:57.326565 8550 heartbeater.cc:503] Master 127.1.47.190:37841 was elected leader, sending a full tablet report...
W20260925 12:21:57.390275 8416 tablet.cc:2369] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:21:57.483700 8279 tablet.cc:2369] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:57.530112 8587 raft_consensus.cc:482] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:21:57.530272 8587 raft_consensus.cc:504] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.531024 8587 leader_election.cc:288] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a584399904fb4e319b34991da3a8317f (127.1.47.129:43517), 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
W20260925 12:21:57.531798 8552 tablet.cc:2369] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:21:57.534921 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.530936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45425 (local address 127.1.47.130:45117)
0925 12:21:57.531279 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.531288 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.531318 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.531651 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.531652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.531653 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.531699 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.531701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.532671 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.532672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.533985 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.533986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.534630 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.534631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.534717 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.534721 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.534789 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.534835 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":437}
I20260925 12:21:57.534921 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.531070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:45117 (local address 127.1.47.131:45425)
0925 12:21:57.531484 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.531486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.531493 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:57.531517 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.531518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.531531 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.531756 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.531791 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.532588 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.534077 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.534541 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.534634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.534644 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.534808 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.534847 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260925 12:21:57.535341 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.530936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43517 (local address 127.1.47.130:54275)
0925 12:21:57.531045 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:57.531058 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:57.531106 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:57.531412 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:57.531413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:57.531417 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:57.531499 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.531501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.532607 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.532609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.533946 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:57.533949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.535034 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.535036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:57.535137 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.535141 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:21:57.535177 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 12:21:57.535225 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":12}
I20260925 12:21:57.535558 8503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98c93d36042a4fda8cbb9287334284aa" candidate_uuid: "872ec0ce067c43e087553da603ca6e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" is_pre_election: true
I20260925 12:21:57.535696 8503 raft_consensus.cc:2454] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872ec0ce067c43e087553da603ca6e02 in term 0.
I20260925 12:21:57.535684 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.531187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:54275 (local address 127.1.47.129:43517)
0925 12:21:57.531254 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.531256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.531265 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:57.531297 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.531315 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.531327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.531599 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:57.531661 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.532509 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.534065 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.534832 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.535513 (+ 681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.535527 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.535550 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.535600 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:21:57.536006 8302 leader_election.cc:302] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53d3e915dee14c3c96ff541f6cd61fa2, 872ec0ce067c43e087553da603ca6e02; no voters:
I20260925 12:21:57.536012 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98c93d36042a4fda8cbb9287334284aa" candidate_uuid: "872ec0ce067c43e087553da603ca6e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a584399904fb4e319b34991da3a8317f" is_pre_election: true
I20260925 12:21:57.536221 8230 raft_consensus.cc:2454] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872ec0ce067c43e087553da603ca6e02 in term 0.
I20260925 12:21:57.536193 8587 raft_consensus.cc:2781] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:21:57.536263 8587 raft_consensus.cc:482] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:21:57.536288 8587 raft_consensus.cc:3037] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:57.537102 8587 raft_consensus.cc:504] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.537251 8587 leader_election.cc:288] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [CANDIDATE]: Term 1 election: Requested vote from peers a584399904fb4e319b34991da3a8317f (127.1.47.129:43517), 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425)
I20260925 12:21:57.537379 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98c93d36042a4fda8cbb9287334284aa" candidate_uuid: "872ec0ce067c43e087553da603ca6e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a584399904fb4e319b34991da3a8317f"
I20260925 12:21:57.537420 8503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98c93d36042a4fda8cbb9287334284aa" candidate_uuid: "872ec0ce067c43e087553da603ca6e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53d3e915dee14c3c96ff541f6cd61fa2"
I20260925 12:21:57.537467 8230 raft_consensus.cc:3037] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:57.537497 8503 raft_consensus.cc:3037] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:21:57.538345 8230 raft_consensus.cc:2454] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872ec0ce067c43e087553da603ca6e02 in term 1.
I20260925 12:21:57.538345 8503 raft_consensus.cc:2454] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872ec0ce067c43e087553da603ca6e02 in term 1.
I20260925 12:21:57.538538 8301 leader_election.cc:302] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872ec0ce067c43e087553da603ca6e02, a584399904fb4e319b34991da3a8317f; no voters:
I20260925 12:21:57.538671 8587 raft_consensus.cc:2781] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:21:57.538836 8587 raft_consensus.cc:686] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 1 LEADER]: Becoming Leader. State: Replica: 872ec0ce067c43e087553da603ca6e02, State: Running, Role: LEADER
I20260925 12:21:57.538938 8587 consensus_queue.cc:237] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:21:57.539923 7312 catalog_manager.cc:5495] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 872ec0ce067c43e087553da603ca6e02 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "872ec0ce067c43e087553da603ca6e02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } health_report { overall_health: UNKNOWN } } }
I20260925 12:21:57.594844 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.592450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.1.47.130:38601)
0925 12:21:57.592535 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.592537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.592547 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:21:57.592591 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.592592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.592634 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.592728 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:21:57.592816 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.593318 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.594078 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.594321 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.594446 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.594547 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:21:57.594671 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:21:57.594741 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.594758 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.594779 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 12:21:57.597056 8503 raft_consensus.cc:1261] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Refusing update from remote peer 872ec0ce067c43e087553da603ca6e02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:57.597056 8230 raft_consensus.cc:1261] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Refusing update from remote peer 872ec0ce067c43e087553da603ca6e02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:21:57.597479 8589 consensus_queue.cc:1035] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:57.597644 8587 consensus_queue.cc:1035] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:21:57.601639 8593 mvcc.cc:205] Tried to move back new op lower bound from 7333228206474072064 to 7333228206240415744. Current Snapshot: MvccSnapshot[applied={T|T < 7333228206474072064}]
I20260925 12:21:57.602855 8594 mvcc.cc:205] Tried to move back new op lower bound from 7333228206474072064 to 7333228206240415744. Current Snapshot: MvccSnapshot[applied={T|T < 7333228206474072064}]
I20260925 12:21:57.603164 7310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:57.603274 7310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:21:57.605023 7310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 273fd950106a4234ad34dea2981ba481: Sending DeleteTablet for 3 replicas of tablet 98c93d36042a4fda8cbb9287334284aa
I20260925 12:21:57.605341 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet 98c93d36042a4fda8cbb9287334284aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:57 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:60954
I20260925 12:21:57.605715 8483 tablet_service.cc:1514] Processing DeleteTablet for tablet 98c93d36042a4fda8cbb9287334284aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:57 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:41788
I20260925 12:21:57.606240 8347 tablet_service.cc:1514] Processing DeleteTablet for tablet 98c93d36042a4fda8cbb9287334284aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:21:57 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:51606
I20260925 12:21:57.606495 8606 tablet_replica.cc:330] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: stopping tablet replica
I20260925 12:21:57.606549 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7280
I20260925 12:21:57.606681 8606 raft_consensus.cc:2229] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:21:57.607708 8606 raft_consensus.cc:2258] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:57.608320 8606 ts_tablet_manager.cc:1905] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:57.610208 8606 ts_tablet_manager.cc:1918] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:57.610349 8606 log.cc:1196] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/98c93d36042a4fda8cbb9287334284aa
I20260925 12:21:57.610731 8606 ts_tablet_manager.cc:1939] T 98c93d36042a4fda8cbb9287334284aa P 872ec0ce067c43e087553da603ca6e02: Deleting consensus metadata
I20260925 12:21:57.612039 8604 tablet_replica.cc:330] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: stopping tablet replica
I20260925 12:21:57.612212 8604 raft_consensus.cc:2229] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:21:57.612376 8604 raft_consensus.cc:2258] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:57.613066 8604 ts_tablet_manager.cc:1905] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:21:57.613123 8301 connection.cc:535] server connection from 127.0.0.1:51606 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 12:21:57.615146 8604 ts_tablet_manager.cc:1918] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:57.615248 8604 log.cc:1196] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/98c93d36042a4fda8cbb9287334284aa
I20260925 12:21:57.615409 8603 tablet_replica.cc:330] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: stopping tablet replica
I20260925 12:21:57.615628 8604 ts_tablet_manager.cc:1939] T 98c93d36042a4fda8cbb9287334284aa P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting consensus metadata
I20260925 12:21:57.615795 8603 raft_consensus.cc:2229] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 12:21:57.616123 8439 connection.cc:441] server connection from 127.0.0.1:41788 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41788 (request call id 1) could send its response
I20260925 12:21:57.616343 8603 raft_consensus.cc:2258] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:21:57.616972 8603 ts_tablet_manager.cc:1905] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:21:57.618916 8603 ts_tablet_manager.cc:1918] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:21:57.619002 8603 log.cc:1196] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/98c93d36042a4fda8cbb9287334284aa
I20260925 12:21:57.619319 8603 ts_tablet_manager.cc:1939] T 98c93d36042a4fda8cbb9287334284aa P a584399904fb4e319b34991da3a8317f: Deleting consensus metadata
W20260925 12:21:57.619745 8164 connection.cc:441] server connection from 127.0.0.1:60954 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60954 (request call id 1) could send its response
I20260925 12:21:57.622326 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.606006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.1.47.188:36409)
0925 12:21:57.606097 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.606099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.612658 (+ 6559us) server_negotiation.cc:407] Connection header received
0925 12:21:57.612774 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.612775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.612793 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.612824 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.613770 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.614543 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.616651 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.616960 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.617006 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.618739 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.618741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.618742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.618748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.620679 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.621167 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.621168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.621169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.621229 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.621421 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.621421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.621422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.621541 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.621916 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.622138 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.622226 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 12:21:57.627137 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.616431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.1.47.188:36409)
0925 12:21:57.616486 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.616488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.616745 (+ 257us) server_negotiation.cc:407] Connection header received
0925 12:21:57.616889 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.616891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.616908 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.616946 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.618328 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.619126 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.620832 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.621192 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.621283 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.623438 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.623440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.623440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.623448 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.625568 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.626209 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.626210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.626211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.626284 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.626497 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.626498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.626499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.626660 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.626781 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.626903 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.627006 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:21:57.628850 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:57.620147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.1.47.189:39049)
0925 12:21:57.620203 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:57.620206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:57.620218 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:21:57.620300 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:57.620301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:57.620316 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:57.620413 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:21:57.621640 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.622488 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.623633 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:57.623990 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:57.624078 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:57.625663 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:21:57.625665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:21:57.625665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:21:57.625673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:21:57.627547 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.628065 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.628067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.628067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.628129 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:21:57.628362 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:21:57.628363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:21:57.628363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:21:57.628489 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:21:57.628586 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:57.628717 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 12:21:57.628788 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 12:21:58.602442 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:58.602118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:37205)
0925 12:21:58.602288 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:58.602373 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260925 12:21:58.602610 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:58.602384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:57973)
0925 12:21:58.602502 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:58.602574 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":49,"threads_started":1}
W20260925 12:21:58.602653 8277 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37841 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
W20260925 12:21:58.602737 8414 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37841 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:58.605825 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:58.605627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:36477)
0925 12:21:58.605728 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:58.605787 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":44,"threads_started":1}
W20260925 12:21:58.606105 8550 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37841 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.105558 8616 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 273fd950106a4234ad34dea2981ba481)
I20260925 12:21:59.105700 8616 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:21:59.106091 8617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 273fd950106a4234ad34dea2981ba481)
I20260925 12:21:59.106335 8616 leader_election.cc:288] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409)
I20260925 12:21:59.106276 8617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:21:59.106483 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.106157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36502)
0925 12:21:59.106388 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.106444 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":55,"threads_started":1}
W20260925 12:21:59.106658 7897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.106917 8617 leader_election.cc:288] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), a5ee8a1711044aea950df8a6599c351d (127.1.47.189:39049)
I20260925 12:21:59.107102 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.106967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36546)
0925 12:21:59.107032 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.107070 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 12:21:59.107198 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.106752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36530)
0925 12:21:59.107131 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.107160 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":50,"threads_started":1}
W20260925 12:21:59.107190 7897 leader_election.cc:334] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
W20260925 12:21:59.107373 8084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.107846 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.107744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36552)
0925 12:21:59.107796 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.107822 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 12:21:59.107981 8084 leader_election.cc:334] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.110430 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.106250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:36409 (local address 127.0.0.1:35816)
0925 12:21:59.106546 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.106557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:59.106613 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:59.107404 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:59.107405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:59.107408 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:59.107713 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:59.107717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.108659 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.108660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:59.109391 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:59.109392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.110207 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.110208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:59.110265 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:59.110270 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:21:59.110314 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 12:21:59.110360 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":52,"threads_started":1}
I20260925 12:21:59.110507 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.107010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:39049 (local address 127.0.0.1:36524)
0925 12:21:59.107204 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.107213 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:21:59.107265 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:21:59.107829 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:21:59.107830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:21:59.107832 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:21:59.107905 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:59.107908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.108607 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.108609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:59.109482 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:21:59.109483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.110250 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.110251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:21:59.110321 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:59.110324 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:21:59.110369 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 12:21:59.110413 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":46,"threads_started":1}
I20260925 12:21:59.110507 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.106478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35816 (local address 127.1.47.188:36409)
0925 12:21:59.106966 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:59.106968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:59.106977 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:21:59.107293 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:59.107295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:59.107308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:59.107390 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:59.107799 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.108549 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.109482 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.110129 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.110213 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:59.110226 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:59.110335 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 12:21:59.110388 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":43,"threads_started":1}
I20260925 12:21:59.110841 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.107049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.1.47.189:39049)
0925 12:21:59.107696 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0925 12:21:59.107698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:21:59.107705 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:21:59.107723 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:21:59.107723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:21:59.107732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:21:59.107825 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:21:59.107994 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.108534 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.109567 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:21:59.110140 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:21:59.110208 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:21:59.110219 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:21:59.110735 (+ 516us) server_negotiation.cc:299] Negotiation successful
0925 12:21:59.110783 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":79,"threads_started":1}
I20260925 12:21:59.110831 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5ee8a1711044aea950df8a6599c351d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" is_pre_election: true
I20260925 12:21:59.110981 8107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5ee8a1711044aea950df8a6599c351d in term 1.
I20260925 12:21:59.111048 7920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5ee8a1711044aea950df8a6599c351d" is_pre_election: true
I20260925 12:21:59.111148 7896 leader_election.cc:302] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e50c0ab9ca94adfa2aebf02947c1df1, a5ee8a1711044aea950df8a6599c351d; no voters: 273fd950106a4234ad34dea2981ba481
I20260925 12:21:59.111157 7920 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e50c0ab9ca94adfa2aebf02947c1df1 in term 1.
I20260925 12:21:59.111275 8616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 12:21:59.111325 8616 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 273fd950106a4234ad34dea2981ba481)
I20260925 12:21:59.111352 8616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:21:59.111361 8081 leader_election.cc:302] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e50c0ab9ca94adfa2aebf02947c1df1, a5ee8a1711044aea950df8a6599c351d; no voters: 273fd950106a4234ad34dea2981ba481
I20260925 12:21:59.111466 8617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 12:21:59.111515 8617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 273fd950106a4234ad34dea2981ba481)
I20260925 12:21:59.111550 8617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:21:59.112417 8616 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:21:59.112434 8617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:21:59.112578 8616 leader_election.cc:288] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 election: Requested vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409)
I20260925 12:21:59.112799 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.112672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36566)
0925 12:21:59.112718 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.112770 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 12:21:59.112846 7920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5ee8a1711044aea950df8a6599c351d"
I20260925 12:21:59.112812 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5ee8a1711044aea950df8a6599c351d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
I20260925 12:21:59.112938 7920 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e50c0ab9ca94adfa2aebf02947c1df1 in current term 2: Already voted for candidate a5ee8a1711044aea950df8a6599c351d in this term.
I20260925 12:21:59.112938 8107 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a5ee8a1711044aea950df8a6599c351d in current term 2: Already voted for candidate 7e50c0ab9ca94adfa2aebf02947c1df1 in this term.
I20260925 12:21:59.113034 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.112720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36578)
0925 12:21:59.112773 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.112798 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 12:21:59.113307 8617 leader_election.cc:288] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 election: Requested vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), a5ee8a1711044aea950df8a6599c351d (127.1.47.189:39049)
I20260925 12:21:59.113384 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.113278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36586)
0925 12:21:59.113330 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.113359 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 12:21:59.113488 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.113331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36584)
0925 12:21:59.113428 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.113463 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260925 12:21:59.113481 7897 leader_election.cc:334] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.113526 7897 leader_election.cc:302] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5ee8a1711044aea950df8a6599c351d; no voters: 273fd950106a4234ad34dea2981ba481, 7e50c0ab9ca94adfa2aebf02947c1df1
W20260925 12:21:59.113570 8084 leader_election.cc:334] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:21:59.113621 8084 leader_election.cc:302] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e50c0ab9ca94adfa2aebf02947c1df1; no voters: 273fd950106a4234ad34dea2981ba481, a5ee8a1711044aea950df8a6599c351d
I20260925 12:21:59.113669 8616 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 12:21:59.113685 8617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 12:21:59.603439 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.603178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:53445)
0925 12:21:59.603341 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.603383 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260925 12:21:59.603457 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.603255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:60517)
0925 12:21:59.603372 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.603405 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260925 12:21:59.606698 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:21:59.606508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:47495)
0925 12:21:59.606625 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 12:21:59.606653 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260925 12:22:00.604277 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.604045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:53617)
0925 12:22:00.604173 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.604212 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260925 12:22:00.604285 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.604045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:36759)
0925 12:22:00.604190 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.604227 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260925 12:22:00.607465 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.607246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:38391)
0925 12:22:00.607380 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.607424 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
I20260925 12:22:00.930853 8632 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:00.930974 8632 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:22:00.931241 8632 leader_election.cc:288] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409)
I20260925 12:22:00.931651 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5ee8a1711044aea950df8a6599c351d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" is_pre_election: true
I20260925 12:22:00.931743 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.931540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36846)
0925 12:22:00.931666 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.931703 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260925 12:22:00.931775 8107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5ee8a1711044aea950df8a6599c351d in term 2.
I20260925 12:22:00.932025 7896 leader_election.cc:302] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e50c0ab9ca94adfa2aebf02947c1df1, a5ee8a1711044aea950df8a6599c351d; no voters:
I20260925 12:22:00.932134 8632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 12:22:00.932181 8632 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:00.932214 8632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 2 FOLLOWER]: Advancing to term 3
I20260925 12:22:00.932382 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.932217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36854)
0925 12:22:00.932270 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.932318 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 12:22:00.932492 7897 leader_election.cc:334] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:22:00.933053 8632 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:22:00.933193 8632 leader_election.cc:288] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 election: Requested vote from peers 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841), 7e50c0ab9ca94adfa2aebf02947c1df1 (127.1.47.188:36409)
I20260925 12:22:00.933368 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.933265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36858)
0925 12:22:00.933315 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.933346 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 12:22:00.933444 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5ee8a1711044aea950df8a6599c351d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1"
I20260925 12:22:00.933534 8107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 2 FOLLOWER]: Advancing to term 3
I20260925 12:22:00.933729 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.933667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36872)
0925 12:22:00.933692 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.933715 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260925 12:22:00.933825 7897 leader_election.cc:334] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
I20260925 12:22:00.934297 8107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5ee8a1711044aea950df8a6599c351d in term 3.
I20260925 12:22:00.934463 7896 leader_election.cc:302] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e50c0ab9ca94adfa2aebf02947c1df1, a5ee8a1711044aea950df8a6599c351d; no voters: 273fd950106a4234ad34dea2981ba481
I20260925 12:22:00.934540 8632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 3 FOLLOWER]: Leader election won for term 3
I20260925 12:22:00.934636 8632 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [term 3 LEADER]: Becoming Leader. State: Replica: a5ee8a1711044aea950df8a6599c351d, State: Running, Role: LEADER
I20260925 12:22:00.934693 8632 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } }
I20260925 12:22:00.935083 8635 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ee8a1711044aea950df8a6599c351d. Latest consensus state: current_term: 3 leader_uuid: "a5ee8a1711044aea950df8a6599c351d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:22:00.935166 8635 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:00.935364 8637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:00.935926 8637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:00.936079 8637 catalog_manager.cc:1259] Loaded cluster ID: 1869eaf3bc064663b5fcc677050f4381
I20260925 12:22:00.936147 8637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:22:00.936367 8637 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:00.936476 8637 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Loaded TSK: 0
I20260925 12:22:00.936672 8637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:22:00.947394 7910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:22:00.949150 7910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:00.950263 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.950129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36890)
0925 12:22:00.950190 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.950224 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 12:22:00.950274 8107 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [term 3 FOLLOWER]: Refusing update from remote peer a5ee8a1711044aea950df8a6599c351d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 12:22:00.950531 8632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 12:22:00.950620 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.950520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.0.0.1:36898)
0925 12:22:00.950567 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.950591 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:22:00.950685 7897 consensus_peers.cc:487] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d -> Peer 273fd950106a4234ad34dea2981ba481 (127.1.47.190:37841): Couldn't send request to peer 273fd950106a4234ad34dea2981ba481. Status: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 12:22:00.951164 8639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ee8a1711044aea950df8a6599c351d. Latest consensus state: current_term: 3 leader_uuid: "a5ee8a1711044aea950df8a6599c351d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:22:00.951252 8639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:22:00.951637 8638 mvcc.cc:205] Tried to move back new op lower bound from 7333228220210114560 to 7333228220149215232. Current Snapshot: MvccSnapshot[applied={T|T < 7333228220210114560}]
I20260925 12:22:00.951740 8632 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a5ee8a1711044aea950df8a6599c351d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:22:00.951894 8632 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:00.952721 8639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a5ee8a1711044aea950df8a6599c351d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "273fd950106a4234ad34dea2981ba481" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37841 } } peers { permanent_uuid: "a5ee8a1711044aea950df8a6599c351d" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 39049 } attrs { promote: false } } peers { permanent_uuid: "7e50c0ab9ca94adfa2aebf02947c1df1" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 36409 } attrs { promote: false } } }
I20260925 12:22:00.952847 8639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e50c0ab9ca94adfa2aebf02947c1df1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:22:00.957875 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.953943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.1.47.129:43517)
0925 12:22:00.954162 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:00.954164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:00.954174 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:00.954202 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:00.954203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:00.954216 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:00.954459 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:00.954500 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.955349 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.956858 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.957514 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.957596 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.957607 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:00.957751 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 12:22:00.957804 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":53,"threads_started":1}
I20260925 12:22:00.957876 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.953818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43517 (local address 127.0.0.1:33810)
0925 12:22:00.953872 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.953884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:00.953936 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:00.954314 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:00.954315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:00.954317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:00.954389 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.954391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.955485 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.955486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.956759 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.956761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.957604 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.957605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.957654 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.957657 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:00.957743 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:22:00.957791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":34}
I20260925 12:22:00.958250 8210 tablet_service.cc:1467] Processing CreateTablet for tablet f7e096d7be2543d6826706c5cfff889a (DEFAULT_TABLE table=test-table [id=bb089078ad4b49458ccbbea41d7f6b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:00.958415 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e096d7be2543d6826706c5cfff889a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:00.958950 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.953819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:38601 (local address 127.0.0.1:51472)
0925 12:22:00.954021 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.954030 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:00.954069 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:00.954374 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:00.954375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:00.954376 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:00.954434 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.954435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.955485 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.955486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.956767 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.956769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.958695 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.958696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.958759 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.958764 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:22:00.958827 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 12:22:00.958880 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":76,"thread_start_us":95,"threads_started":1}
I20260925 12:22:00.958971 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.954041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.1.47.130:38601)
0925 12:22:00.954209 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:00.954212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:00.954220 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:00.954248 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:00.954248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:00.954290 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:00.954364 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:00.954502 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.955361 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.957647 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.958611 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.958685 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.958699 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:00.958848 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 12:22:00.958893 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":52,"threads_started":1}
I20260925 12:22:00.959401 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.953892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45425 (local address 127.0.0.1:52460)
0925 12:22:00.954233 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.954244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:00.954290 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:00.955644 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:00.955645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:00.955647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:00.955731 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.955732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.956646 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.956647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.957872 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:00.957875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.959112 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.959113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:00.959188 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.959191 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:00.959268 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 12:22:00.959312 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":32,"thread_start_us":67,"threads_started":1}
I20260925 12:22:00.959450 8347 tablet_service.cc:1467] Processing CreateTablet for tablet f7e096d7be2543d6826706c5cfff889a (DEFAULT_TABLE table=test-table [id=bb089078ad4b49458ccbbea41d7f6b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:00.959488 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.954138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.1.47.131:45425)
0925 12:22:00.955496 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:00.955498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:00.955506 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:00.955539 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:00.955540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:00.955548 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:00.955628 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:00.955812 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.956557 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.957972 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:00.958966 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:00.959060 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:00.959075 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:00.959274 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 12:22:00.959313 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":45,"threads_started":1}
I20260925 12:22:00.959555 8347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e096d7be2543d6826706c5cfff889a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:00.960058 8650 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: Bootstrap starting.
I20260925 12:22:00.960425 8650 tablet_bootstrap.cc:654] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:00.960961 8483 tablet_service.cc:1467] Processing CreateTablet for tablet f7e096d7be2543d6826706c5cfff889a (DEFAULT_TABLE table=test-table [id=bb089078ad4b49458ccbbea41d7f6b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:00.960947 8651 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Bootstrap starting.
I20260925 12:22:00.961102 8483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e096d7be2543d6826706c5cfff889a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:00.961587 8651 tablet_bootstrap.cc:654] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:00.961618 8650 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: No bootstrap required, opened a new log
I20260925 12:22:00.961668 8650 ts_tablet_manager.cc:1397] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:00.961843 8650 raft_consensus.cc:348] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.961928 8650 raft_consensus.cc:374] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:00.961946 8650 raft_consensus.cc:729] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a584399904fb4e319b34991da3a8317f, State: Initialized, Role: FOLLOWER
I20260925 12:22:00.962011 8650 consensus_queue.cc:260] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.962200 8650 ts_tablet_manager.cc:1428] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:00.962469 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.962359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:47673)
0925 12:22:00.962408 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.962437 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 12:22:00.962715 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.962650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:35641)
0925 12:22:00.962678 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.962696 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 12:22:00.962836 8278 heartbeater.cc:503] Master 127.1.47.189:39049 was elected leader, sending a full tablet report...
I20260925 12:22:00.962872 8655 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Bootstrap starting.
I20260925 12:22:00.963120 8651 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: No bootstrap required, opened a new log
I20260925 12:22:00.963268 8651 ts_tablet_manager.cc:1397] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:00.963403 8651 raft_consensus.cc:348] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.963487 8655 tablet_bootstrap.cc:654] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:00.963496 8651 raft_consensus.cc:374] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:00.963522 8651 raft_consensus.cc:729] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872ec0ce067c43e087553da603ca6e02, State: Initialized, Role: FOLLOWER
I20260925 12:22:00.963586 8651 consensus_queue.cc:260] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.963788 8651 ts_tablet_manager.cc:1428] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:00.964148 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.964037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:39593)
0925 12:22:00.964081 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.964116 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 12:22:00.964499 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.964403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:54859)
0925 12:22:00.964443 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.964469 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 12:22:00.964583 8415 heartbeater.cc:503] Master 127.1.47.189:39049 was elected leader, sending a full tablet report...
I20260925 12:22:00.965258 8655 tablet_bootstrap.cc:492] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: No bootstrap required, opened a new log
I20260925 12:22:00.965306 8655 ts_tablet_manager.cc:1397] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:00.965435 8655 raft_consensus.cc:348] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.965490 8655 raft_consensus.cc:374] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:00.965505 8655 raft_consensus.cc:729] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d3e915dee14c3c96ff541f6cd61fa2, State: Initialized, Role: FOLLOWER
I20260925 12:22:00.965574 8655 consensus_queue.cc:260] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:00.965782 8655 ts_tablet_manager.cc:1428] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:00.966074 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.965951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:45007)
0925 12:22:00.965985 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.966015 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 12:22:00.966259 8551 heartbeater.cc:503] Master 127.1.47.189:39049 was elected leader, sending a full tablet report...
I20260925 12:22:00.966565 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:00.966467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:44303)
0925 12:22:00.966504 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:00.966535 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 12:22:01.097368 8658 raft_consensus.cc:482] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:01.097479 8658 raft_consensus.cc:504] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:01.097786 8658 leader_election.cc:288] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a584399904fb4e319b34991da3a8317f (127.1.47.129:43517), 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:22:01.100998 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.098119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:48609 (local address 127.1.47.130:38601)
0925 12:22:01.098170 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:01.098172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:01.098228 (+ 56us) server_negotiation.cc:407] Connection header received
0925 12:22:01.098258 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:01.098260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:01.098275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:01.098334 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:01.098535 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.099084 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100256 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.100631 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100730 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:01.100741 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:01.100888 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 12:22:01.100926 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:22:01.101011 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.098037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:38601 (local address 127.1.47.131:48609)
0925 12:22:01.098100 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.098113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:01.098169 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:01.098382 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:01.098384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:01.098387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:01.098452 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:01.098454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.099174 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.099175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:01.100154 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:01.100157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.100730 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:01.100801 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:01.100805 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:01.100874 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 12:22:01.100916 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260925 12:22:01.101023 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.098119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:53801 (local address 127.1.47.129:43517)
0925 12:22:01.098180 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:01.098182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:01.098372 (+ 190us) server_negotiation.cc:407] Connection header received
0925 12:22:01.098403 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:01.098404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:01.098416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:01.098479 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:01.098683 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.099208 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100297 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.100670 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100764 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:01.100774 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:01.100918 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 12:22:01.100952 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:22:01.101104 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.098037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43517 (local address 127.1.47.131:53801)
0925 12:22:01.098255 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.098266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:01.098328 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:01.098535 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:01.098536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:01.098538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:01.098578 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:01.098580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.099308 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.099309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:01.100210 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:01.100212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.100766 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.100767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:01.100834 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:01.100837 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:01.100910 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 12:22:01.100950 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":21,"thread_start_us":126,"threads_started":1}
I20260925 12:22:01.101346 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e096d7be2543d6826706c5cfff889a" candidate_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a584399904fb4e319b34991da3a8317f" is_pre_election: true
I20260925 12:22:01.101475 8230 raft_consensus.cc:2454] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53d3e915dee14c3c96ff541f6cd61fa2 in term 0.
I20260925 12:22:01.101536 8367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e096d7be2543d6826706c5cfff889a" candidate_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872ec0ce067c43e087553da603ca6e02" is_pre_election: true
I20260925 12:22:01.101698 8367 raft_consensus.cc:2454] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53d3e915dee14c3c96ff541f6cd61fa2 in term 0.
I20260925 12:22:01.101730 8437 leader_election.cc:302] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53d3e915dee14c3c96ff541f6cd61fa2, a584399904fb4e319b34991da3a8317f; no voters:
I20260925 12:22:01.101877 8658 raft_consensus.cc:2781] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:01.101926 8658 raft_consensus.cc:482] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:01.101958 8658 raft_consensus.cc:3037] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:01.102681 8658 raft_consensus.cc:504] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:01.102843 8658 leader_election.cc:288] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [CANDIDATE]: Term 1 election: Requested vote from peers a584399904fb4e319b34991da3a8317f (127.1.47.129:43517), 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601)
I20260925 12:22:01.102985 8367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e096d7be2543d6826706c5cfff889a" candidate_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872ec0ce067c43e087553da603ca6e02"
I20260925 12:22:01.103052 8367 raft_consensus.cc:3037] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:01.103021 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e096d7be2543d6826706c5cfff889a" candidate_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a584399904fb4e319b34991da3a8317f"
I20260925 12:22:01.103111 8230 raft_consensus.cc:3037] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:01.103713 8367 raft_consensus.cc:2454] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53d3e915dee14c3c96ff541f6cd61fa2 in term 1.
I20260925 12:22:01.103729 8230 raft_consensus.cc:2454] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53d3e915dee14c3c96ff541f6cd61fa2 in term 1.
I20260925 12:22:01.103906 8436 leader_election.cc:302] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53d3e915dee14c3c96ff541f6cd61fa2, 872ec0ce067c43e087553da603ca6e02; no voters:
I20260925 12:22:01.104038 8658 raft_consensus.cc:2781] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:01.104197 8658 raft_consensus.cc:686] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 LEADER]: Becoming Leader. State: Replica: 53d3e915dee14c3c96ff541f6cd61fa2, State: Running, Role: LEADER
I20260925 12:22:01.104279 8658 consensus_queue.cc:237] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } }
I20260925 12:22:01.104846 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.104542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:35929)
0925 12:22:01.104593 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.104810 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 12:22:01.105149 7910 catalog_manager.cc:5495] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53d3e915dee14c3c96ff541f6cd61fa2" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45425 } health_report { overall_health: HEALTHY } } }
I20260925 12:22:01.140591 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.137906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.1.47.131:45425)
0925 12:22:01.137970 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:01.137971 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:01.137981 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:01.138028 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:01.138028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:01.138050 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:01.138120 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:01.138284 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.138780 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.139783 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:01.140007 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:01.140072 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:01.140255 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:01.140383 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:01.140463 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:01.140511 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 12:22:01.140532 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:22:01.141973 8230 raft_consensus.cc:1261] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f [term 1 FOLLOWER]: Refusing update from remote peer 53d3e915dee14c3c96ff541f6cd61fa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:01.142042 8367 raft_consensus.cc:1261] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Refusing update from remote peer 53d3e915dee14c3c96ff541f6cd61fa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:01.142370 8658 consensus_queue.cc:1035] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "872ec0ce067c43e087553da603ca6e02" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:01.142532 8660 consensus_queue.cc:1035] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a584399904fb4e319b34991da3a8317f" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:01.143503 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.143356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:52405)
0925 12:22:01.143426 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.143469 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 12:22:01.143610 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.143455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:55773)
0925 12:22:01.143549 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.143577 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 12:22:01.145228 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.145108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.131:57283)
0925 12:22:01.145147 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.145186 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 12:22:01.145432 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.145337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.129:33147)
0925 12:22:01.145368 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.145395 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 12:22:01.145531 8666 mvcc.cc:205] Tried to move back new op lower bound from 7333228220994727936 to 7333228220843773952. Current Snapshot: MvccSnapshot[applied={T|T < 7333228220994727936}]
I20260925 12:22:01.145771 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.145677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37841 (local address 127.1.47.130:59775)
0925 12:22:01.145718 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.145743 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 12:22:01.148190 7910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:01.148283 7910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:01.149695 7910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5ee8a1711044aea950df8a6599c351d: Sending DeleteTablet for 3 replicas of tablet f7e096d7be2543d6826706c5cfff889a
I20260925 12:22:01.150000 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e096d7be2543d6826706c5cfff889a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:01 UTC) from {username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:33810
I20260925 12:22:01.150039 8347 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e096d7be2543d6826706c5cfff889a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:01 UTC) from {username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:51472
I20260925 12:22:01.150559 8483 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e096d7be2543d6826706c5cfff889a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:01 UTC) from {username='oryx', principal='oryx/127.1.47.189@KRBTEST.COM'} at 127.0.0.1:52460
I20260925 12:22:01.150591 8674 tablet_replica.cc:330] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: stopping tablet replica
I20260925 12:22:01.150651 8674 raft_consensus.cc:2229] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:01.150700 8674 raft_consensus.cc:2258] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:01.150753 8676 tablet_replica.cc:330] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: stopping tablet replica
I20260925 12:22:01.150810 8676 raft_consensus.cc:2229] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:22:01.150905 8676 raft_consensus.cc:2258] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:01.150905 8675 tablet_replica.cc:330] T f7e096d7be2543d6826706c5cfff889a P a584399904fb4e319b34991da3a8317f: stopping tablet replica
I20260925 12:22:01.150889 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8059
I20260925 12:22:01.151154 8674 ts_tablet_manager.cc:1905] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:01.151672 8676 ts_tablet_manager.cc:1905] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:01.153342 8674 ts_tablet_manager.cc:1918] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:01.153342 8676 ts_tablet_manager.cc:1918] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:01.153394 8676 log.cc:1196] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/f7e096d7be2543d6826706c5cfff889a
I20260925 12:22:01.153394 8674 log.cc:1196] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/f7e096d7be2543d6826706c5cfff889a
I20260925 12:22:01.153648 8674 ts_tablet_manager.cc:1939] T f7e096d7be2543d6826706c5cfff889a P 872ec0ce067c43e087553da603ca6e02: Deleting consensus metadata
I20260925 12:22:01.153648 8676 ts_tablet_manager.cc:1939] T f7e096d7be2543d6826706c5cfff889a P 53d3e915dee14c3c96ff541f6cd61fa2: Deleting consensus metadata
I20260925 12:22:01.153962 7894 catalog_manager.cc:4841] TS 872ec0ce067c43e087553da603ca6e02 (127.1.47.130:38601): tablet f7e096d7be2543d6826706c5cfff889a (table test-table [id=bb089078ad4b49458ccbbea41d7f6b8f]) successfully deleted
I20260925 12:22:01.153976 7896 catalog_manager.cc:4841] TS 53d3e915dee14c3c96ff541f6cd61fa2 (127.1.47.131:45425): tablet f7e096d7be2543d6826706c5cfff889a (table test-table [id=bb089078ad4b49458ccbbea41d7f6b8f]) successfully deleted
I20260925 12:22:01.159181 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:01.159053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43517 (local address 127.0.0.1:33822)
0925 12:22:01.159113 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:01.159140 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 12:22:01.159240 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8284
W20260925 12:22:01.159363 7895 catalog_manager.cc:4584] TS a584399904fb4e319b34991da3a8317f (127.1.47.129:43517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7e096d7be2543d6826706c5cfff889a: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43517: connect: Connection refused (error 111)
I20260925 12:22:01.165228 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8420
I20260925 12:22:01.170116 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7776
I20260925 12:22:01.175094 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 7964
2026-09-25T12:22:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6864 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:22:01 dist-test-slave-l9ww krb5kdc[8679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:22:01 dist-test-slave-l9ww krb5kdc[8679](info): set up 2 sockets
Sep 25 12:22:01 dist-test-slave-l9ww krb5kdc[8679](info): commencing operation
krb5kdc: starting...
W20260925 12:22:02.216863 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab.
Entry for principal oryx/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab.
I20260925 12:22:02.341308 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43525
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab
--principal=oryx/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:43525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:02.424683 8692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:02.424983 8692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:02.425017 8692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:02.426770 8692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:02.426851 8692 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 12:22:02.426870 8692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:02.426894 8692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:02.426972 8692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:02.427006 8692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:02.429132 8692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:43525
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.190.keytab
--principal=oryx/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:02.429414 8692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:02.429670 8692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:02.434937 8697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.435012 8692 server_base.cc:1034] running on GCE node
W20260925 12:22:02.434937 8700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.434975 8698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.435385 8692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:02.435701 8692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:02.436878 8692 hybrid_clock.cc:648] HybridClock initialized: now 1790338922436863 us; error 37 us; skew 500 ppm
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:02.448585 8692 init.cc:377] Logged in from keytab as oryx/127.1.47.190@KRBTEST.COM (short username oryx)
I20260925 12:22:02.449047 8692 webserver.cc:460] Webserver started at http://127.1.47.190:37609/ using document root <none> and password file <none>
I20260925 12:22:02.449293 8692 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:02.449362 8692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:02.449517 8692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:02.450618 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "912332e85a9c4313b2876ffd60bf9d45"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "ceeae7f7b0c2348e4c0ec97efa47a878"
server_key_iv: "4c21c5cae5a283282aee4ee56044f4a0"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.451066 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "912332e85a9c4313b2876ffd60bf9d45"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "ceeae7f7b0c2348e4c0ec97efa47a878"
server_key_iv: "4c21c5cae5a283282aee4ee56044f4a0"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.452581 8692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:22:02.453430 8707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.453660 8692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:02.453781 8692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "912332e85a9c4313b2876ffd60bf9d45"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "ceeae7f7b0c2348e4c0ec97efa47a878"
server_key_iv: "4c21c5cae5a283282aee4ee56044f4a0"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.453882 8692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:02.479642 8692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:02.482008 8692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:02.482203 8692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:02.486994 8692 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:43525
I20260925 12:22:02.487046 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:43525 every 8 connection(s)
I20260925 12:22:02.487396 8692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:02.488125 8770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:02.490439 8770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Bootstrap starting.
I20260925 12:22:02.491050 8770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:02.491356 8770 log.cc:824] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:02.492091 8770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: No bootstrap required, opened a new log
I20260925 12:22:02.494163 8770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } }
I20260925 12:22:02.494309 8770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:02.494331 8770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912332e85a9c4313b2876ffd60bf9d45, State: Initialized, Role: FOLLOWER
I20260925 12:22:02.494511 8770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } }
I20260925 12:22:02.494576 8770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:02.494597 8770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:02.494663 8770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:02.495232 8770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } }
I20260925 12:22:02.495345 8770 leader_election.cc:302] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912332e85a9c4313b2876ffd60bf9d45; no voters:
I20260925 12:22:02.495548 8770 leader_election.cc:288] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:22:02.495664 8773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:02.495824 8773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [term 1 LEADER]: Becoming Leader. State: Replica: 912332e85a9c4313b2876ffd60bf9d45, State: Running, Role: LEADER
I20260925 12:22:02.495836 8770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:02.495960 8773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } }
I20260925 12:22:02.496347 8774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912332e85a9c4313b2876ffd60bf9d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } } }
I20260925 12:22:02.496423 8774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:02.496585 8775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912332e85a9c4313b2876ffd60bf9d45. Latest consensus state: current_term: 1 leader_uuid: "912332e85a9c4313b2876ffd60bf9d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912332e85a9c4313b2876ffd60bf9d45" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43525 } } }
I20260925 12:22:02.496634 8775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45 [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:02.496762 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8692
I20260925 12:22:02.496788 8780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:02.496858 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:22:02.497049 1214 external_mini_cluster.cc:1428] Setting key e4c0cddd9ae81ea46624e354d06d8252
I20260925 12:22:02.497268 8780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:02.498836 8780 catalog_manager.cc:1347] Generated new cluster ID: e8efec5c2466430aab3993ca46dc3323
I20260925 12:22:02.498895 8780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:02.507220 8780 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:22:02.507766 8780 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:02.509441 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.497525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.1.47.190:43525)
0925 12:22:02.497982 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.497991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.498026 (+ 35us) server_negotiation.cc:407] Connection header received
0925 12:22:02.498146 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.498153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.498405 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.498545 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.499431 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.500850 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.501956 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.502608 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.502704 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.505603 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.505608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.505608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.505622 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.507701 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.508175 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.508178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.508179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.508292 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.508507 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.508508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.508508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.508796 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.508977 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.509047 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.509198 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":72,"threads_started":1}
I20260925 12:22:02.512705 8780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Generated new TSK 0
I20260925 12:22:02.512910 8780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
I20260925 12:22:02.543625 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43525
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:02.627104 8796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:02.627336 8796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:02.627362 8796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:02.628635 8796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:02.628682 8796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:02.628705 8796 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:22:02.628723 8796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:02.630249 8796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:02.630470 8796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:02.630658 8796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:02.635957 8801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.635998 8804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.636125 8802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.636051 8796 server_base.cc:1034] running on GCE node
I20260925 12:22:02.636376 8796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:02.636592 8796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:02.637784 8796 hybrid_clock.cc:648] HybridClock initialized: now 1790338922637773 us; error 35 us; skew 500 ppm
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:02.641041 8796 init.cc:377] Logged in from keytab as oryx/127.1.47.129@KRBTEST.COM (short username oryx)
I20260925 12:22:02.641614 8796 webserver.cc:460] Webserver started at http://127.1.47.129:37411/ using document root <none> and password file <none>
I20260925 12:22:02.641883 8796 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:02.641964 8796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:02.642143 8796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:02.643356 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a5f7cd9399e2cb36b277e412c8674724"
server_key_iv: "3d2944aba30d86ac219a7f1d38da8155"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.643806 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a5f7cd9399e2cb36b277e412c8674724"
server_key_iv: "3d2944aba30d86ac219a7f1d38da8155"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.645421 8796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:02.646301 8811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.646521 8796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:02.646658 8796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a5f7cd9399e2cb36b277e412c8674724"
server_key_iv: "3d2944aba30d86ac219a7f1d38da8155"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.646749 8796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:02.656651 8796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:02.658733 8796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:02.658898 8796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:02.659438 8796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:02.659507 8796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.659548 8796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:02.659576 8796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:02.666620 8796 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:34863
I20260925 12:22:02.666692 8926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:34863 every 8 connection(s)
I20260925 12:22:02.666994 8796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:22:02.668821 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8796
I20260925 12:22:02.668933 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:22:02.669123 1214 external_mini_cluster.cc:1428] Setting key 8fdde7b9b3c8e11c985dce38e24d6d0e
I20260925 12:22:02.670475 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.659934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:55957 (local address 127.1.47.190:43525)
0925 12:22:02.660004 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.660007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.660244 (+ 237us) server_negotiation.cc:407] Connection header received
0925 12:22:02.660749 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.660751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.660766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.660863 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.661467 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.662326 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.663982 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.664337 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.664420 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.667133 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.667135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.667135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.667143 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.668867 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.669419 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.669421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.669421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.669484 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.669762 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.669763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.669763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.669900 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.669985 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.670243 (+ 258us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.670336 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:22:02.670511 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.659969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.129:55957)
0925 12:22:02.660110 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.660150 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.660650 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.660937 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.660941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.661177 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.661359 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.661364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.662428 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.662429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.663861 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.663865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.664448 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.664449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.664608 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.665108 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:02.665128 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:02.667031 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:02.669012 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.669018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.669022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.669335 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.669559 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.669560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.669560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.669690 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.669989 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:02.669994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:02.670109 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 12:22:02.670198 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 12:22:02.670315 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260925 12:22:02.671708 8927 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43525
I20260925 12:22:02.671797 8927 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:02.672008 8927 heartbeater.cc:511] Master 127.1.47.190:43525 requested a full tablet report, sending...
I20260925 12:22:02.672570 8724 ts_manager.cc:194] Registered new tserver with Master: 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863)
I20260925 12:22:02.673290 8724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:55957
WARNING: no policy specified for oryx/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.130@KRBTEST.COM" created.
I20260925 12:22:02.683409 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.672810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.129:53099)
0925 12:22:02.675993 (+ 3183us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.676005 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.676082 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.676229 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.676229 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.676445 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.676503 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.676505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.677500 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.677501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.678133 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.678135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.678509 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.678510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.678606 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.679071 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:02.679076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:02.679347 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:02.682142 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.682143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.682144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.682427 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.682656 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.682656 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.682657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.682756 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.683105 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:02.683106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:02.683153 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 12:22:02.683228 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 12:22:02.683320 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3158,"thread_start_us":74,"threads_started":1}
I20260925 12:22:02.683493 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.672874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:53099 (local address 127.1.47.190:43525)
0925 12:22:02.672919 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.672921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.676092 (+ 3171us) server_negotiation.cc:407] Connection header received
0925 12:22:02.676140 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.676141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.676151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.676215 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.676590 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.677395 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.678210 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.678395 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.678461 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.679458 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.679460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.679460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.679465 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.682036 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.682509 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.682510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.682510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.682559 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.682831 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.682832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.682832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.682996 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.683067 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.683272 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.683360 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
I20260925 12:22:02.697248 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43525
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:02.778602 8936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:02.778820 8936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:02.778844 8936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:02.780001 8936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:02.780037 8936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:02.780057 8936 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:22:02.780074 8936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:02.781533 8936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:02.781796 8936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:02.781981 8936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:02.787230 8944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.787263 8942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.787276 8941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.787570 8936 server_base.cc:1034] running on GCE node
I20260925 12:22:02.787781 8936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:02.788081 8936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:02.789264 8936 hybrid_clock.cc:648] HybridClock initialized: now 1790338922789255 us; error 41 us; skew 500 ppm
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:02.792135 8936 init.cc:377] Logged in from keytab as oryx/127.1.47.130@KRBTEST.COM (short username oryx)
I20260925 12:22:02.792548 8936 webserver.cc:460] Webserver started at http://127.1.47.130:34139/ using document root <none> and password file <none>
I20260925 12:22:02.792750 8936 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:02.792793 8936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:02.792902 8936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:02.793992 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "4f3a48986ba0476986acf81f930812f6"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "d17219314750054cfc873008daade3d4"
server_key_iv: "266983b0f8147570994f2ddde7e4dd93"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.794310 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "4f3a48986ba0476986acf81f930812f6"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "d17219314750054cfc873008daade3d4"
server_key_iv: "266983b0f8147570994f2ddde7e4dd93"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.795694 8936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:22:02.796535 8951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.796690 8936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:22:02.796764 8936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "4f3a48986ba0476986acf81f930812f6"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "d17219314750054cfc873008daade3d4"
server_key_iv: "266983b0f8147570994f2ddde7e4dd93"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.796818 8936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:02.825922 8936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:02.828508 8936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:02.828624 8936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:02.829085 8936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:02.829123 8936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.829144 8936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:02.829157 8936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.836306 8936 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:41101
I20260925 12:22:02.836360 9066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:41101 every 8 connection(s)
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:02.836742 8936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:22:02.840281 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.829770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:44821 (local address 127.1.47.190:43525)
0925 12:22:02.829846 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.829849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.830049 (+ 200us) server_negotiation.cc:407] Connection header received
0925 12:22:02.830752 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.830753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.830769 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.830852 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.831845 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.832748 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.833922 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.834154 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.834250 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.837113 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.837115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.837115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.837122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.838552 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.839045 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.839046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.839047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.839123 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.839432 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.839433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.839434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.839577 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.839682 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.840081 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.840176 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 12:22:02.840517 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.829722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.130:44821)
0925 12:22:02.829910 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.829956 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.830556 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.830962 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.830968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.831387 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.831715 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.831721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.832865 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.832866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.833775 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.833777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.834263 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.834264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.834419 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.835048 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:02.835060 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:02.836989 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:02.838667 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.838675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.838677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.838937 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.839203 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.839205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.839205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.839348 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.839675 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:02.839681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:02.839842 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 12:22:02.840054 (+ 212us) client_negotiation.cc:240] Negotiation successful
0925 12:22:02.840234 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260925 12:22:02.841044 9067 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43525
I20260925 12:22:02.841149 9067 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:02.841331 9067 heartbeater.cc:511] Master 127.1.47.190:43525 requested a full tablet report, sending...
I20260925 12:22:02.842181 8723 ts_manager.cc:194] Registered new tserver with Master: 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101)
I20260925 12:22:02.842646 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 8936
I20260925 12:22:02.842722 8723 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:44821
I20260925 12:22:02.842759 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:22:02.842968 1214 external_mini_cluster.cc:1428] Setting key fb58331b6d7a2f66d6ad1a22f087c9fe
I20260925 12:22:02.848611 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.842147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:35667 (local address 127.1.47.190:43525)
0925 12:22:02.842188 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.842190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.842287 (+ 97us) server_negotiation.cc:407] Connection header received
0925 12:22:02.842371 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.842372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.842385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.842483 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.843071 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.843654 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.844427 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.844792 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.844882 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.845982 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.845983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.845983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.845990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.847445 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.847782 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.847783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.847783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.847842 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.848069 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.848070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.848071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.848211 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.848284 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.848455 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.848526 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:22:02.848611 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.842107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.130:35667)
0925 12:22:02.842230 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.842239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.842310 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.842489 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.842491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.842707 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.842783 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.842785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.843737 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.843738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.844338 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.844340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.844877 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.844878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.844936 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.845469 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:02.845474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:02.845865 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:02.847526 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.847528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.847528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.847707 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.847939 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.847939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.847940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.848006 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.848288 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:02.848289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:02.848333 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 12:22:02.848412 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 12:22:02.848516 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
I20260925 12:22:02.871992 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43525
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:02.956852 9076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:02.957060 9076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:02.957083 9076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:02.958197 9076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:02.958231 9076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:02.958252 9076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 12:22:02.958269 9076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:02.959709 9076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.47.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:02.959972 9076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:02.960160 9076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:02.965474 9082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.965608 9076 server_base.cc:1034] running on GCE node
W20260925 12:22:02.965490 9084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:02.965535 9081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:02.965906 9076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:02.966168 9076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:02.967361 9076 hybrid_clock.cc:648] HybridClock initialized: now 1790338922967336 us; error 58 us; skew 500 ppm
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:02.970006 9076 init.cc:377] Logged in from keytab as oryx/127.1.47.131@KRBTEST.COM (short username oryx)
I20260925 12:22:02.970494 9076 webserver.cc:460] Webserver started at http://127.1.47.131:35001/ using document root <none> and password file <none>
I20260925 12:22:02.970728 9076 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:02.970794 9076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:02.970940 9076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:02.972172 9076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a8238293c13ce15183f3121764e8ca9a"
server_key_iv: "66f0d351865969aa32011f42166e53cb"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.972533 9076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a8238293c13ce15183f3121764e8ca9a"
server_key_iv: "66f0d351865969aa32011f42166e53cb"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.973896 9076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:22:02.974691 9091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.974874 9076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:02.974982 9076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
format_stamp: "Formatted at 2026-09-25 12:22:02 on dist-test-slave-l9ww"
server_key: "a8238293c13ce15183f3121764e8ca9a"
server_key_iv: "66f0d351865969aa32011f42166e53cb"
server_key_version: "encryptionkey@0"
I20260925 12:22:02.975065 9076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:02.981793 9076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:02.984171 9076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:02.984323 9076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:02.984853 9076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:02.984910 9076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:02.984957 9076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:02.984983 9076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 12:22:02 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:02.992244 9076 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:43519
I20260925 12:22:02.992329 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:43519 every 8 connection(s)
I20260925 12:22:02.992707 9076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:22:02.994745 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.985254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:59261 (local address 127.1.47.190:43525)
0925 12:22:02.985316 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.985319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.985554 (+ 235us) server_negotiation.cc:407] Connection header received
0925 12:22:02.986034 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.986035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.986050 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.986135 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.986875 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.987696 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.989432 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.989699 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.989785 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.992002 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:02.992004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:02.992004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:02.992012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:02.993601 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.993980 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.993981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.993981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.994033 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:02.994236 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:02.994236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:02.994237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:02.994338 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:02.994386 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:02.994620 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 12:22:02.994683 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:22:02.994977 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.985249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.131:59261)
0925 12:22:02.985418 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.985457 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.985942 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.986222 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.986226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.986533 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.986769 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.986774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.987811 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.987812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.989313 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.989317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.989805 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.989806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.989934 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:02.990469 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:02.990478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:02.991838 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:02.993675 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.993680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.993681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.993892 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.994088 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:02.994089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:02.994089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:02.994181 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:02.994390 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:02.994394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:02.994509 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 12:22:02.994593 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:22:02.994713 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
I20260925 12:22:02.995476 9207 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43525
I20260925 12:22:02.995559 9207 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:02.995746 9207 heartbeater.cc:511] Master 127.1.47.190:43525 requested a full tablet report, sending...
I20260925 12:22:02.996165 8724 ts_manager.cc:194] Registered new tserver with Master: 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:02.996666 8724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:59261
I20260925 12:22:02.997020 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9076
I20260925 12:22:02.997165 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:22:02.997375 1214 external_mini_cluster.cc:1428] Setting key 8209a8b9eb16cb7ba9d9383d4ec2e0b0
I20260925 12:22:02.998212 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:22:03.003600 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.996531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.1.47.131:55507)
0925 12:22:02.996654 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:02.996667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:02.996719 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:02.996914 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:02.996915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:02.997148 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:02.997218 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.997220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.998239 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.998240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.999064 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:02.999067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.999521 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.999522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:02.999596 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.000186 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:03.000192 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:03.000536 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:03.002426 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:03.002427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:03.002427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:03.002606 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:03.002862 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:03.002863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:03.002864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:03.002964 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:03.003288 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:03.003289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:03.003334 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.003399 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.003501 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
I20260925 12:22:03.003639 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:02.996621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:55507 (local address 127.1.47.190:43525)
0925 12:22:02.996741 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:02.996744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:02.996762 (+ 18us) server_negotiation.cc:407] Connection header received
0925 12:22:02.996782 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:02.996783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:02.996791 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:02.996841 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:02.997292 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.998076 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.999171 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:02.999389 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:02.999522 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.000628 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:03.000630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:03.000630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:03.000638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:03.002317 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.002684 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.002686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.002686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.002750 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.003056 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.003057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.003057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.003202 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:03.003276 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.003453 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.003542 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 12:22:03.021435 8786 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:03.015541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.1.47.190:43525)
0925 12:22:03.015600 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.015602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.015619 (+ 17us) server_negotiation.cc:407] Connection header received
0925 12:22:03.015644 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.015644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.015653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.015712 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:03.016503 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.016998 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.017849 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.018055 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.018115 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.021278 (+ 3163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34938: BlockingRecv error: recv got EOF from 127.0.0.1:34938 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:03.029954 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.022336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.1.47.190:43525)
0925 12:22:03.022401 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.022403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.022516 (+ 113us) server_negotiation.cc:407] Connection header received
0925 12:22:03.022596 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.022597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.022607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.022663 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:03.023151 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.023626 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.024410 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.024609 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.024692 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.026944 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:03.026945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:03.026945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:03.026952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:03.028567 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.029008 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.029009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.029010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.029081 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.029340 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.029341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.029341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.029472 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:03.029598 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.029730 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.029832 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 12:22:03.031963 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:22:03.033582 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:03.040445 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34863 (local address 127.0.0.1:44576)
0925 12:22:03.036556 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.036565 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.036630 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.036974 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.036977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.036980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.037063 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.037065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.038036 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.038037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.039194 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.039196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.040110 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.040111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.040203 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.040217 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.040294 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.040348 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":105,"threads_started":1}
I20260925 12:22:03.040459 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.1.47.129:34863)
0925 12:22:03.036801 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.036805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.036814 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:03.036853 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.036855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.036869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.036949 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.037152 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.037947 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.039276 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.040012 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.040108 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.040194 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.040309 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.040348 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":69,"threads_started":1}
I20260925 12:22:03.040690 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.0.0.1:55304)
0925 12:22:03.036323 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.036363 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.036463 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.036999 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.037000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.037003 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.037069 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.037071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.038545 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.038546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.039500 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.039501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.040485 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.040486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.040547 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.040551 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.040580 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.040616 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260925 12:22:03.041040 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.1.47.130:41101)
0925 12:22:03.036741 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.036745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.036771 (+ 26us) server_negotiation.cc:407] Connection header received
0925 12:22:03.036804 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.036805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.036818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.036935 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.037334 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.038447 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.039592 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.040384 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.040769 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.040871 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.040899 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.040944 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":77,"threads_started":1}
I20260925 12:22:03.041656 9000 tablet_service.cc:1467] Processing CreateTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e (DEFAULT_TABLE table=test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.041656 8861 tablet_service.cc:1467] Processing CreateTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e (DEFAULT_TABLE table=test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.042044 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec0fa4f7dbe84e949570e58a505ab67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.042044 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec0fa4f7dbe84e949570e58a505ab67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.046139 9236 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Bootstrap starting.
I20260925 12:22:03.046224 9235 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Bootstrap starting.
I20260925 12:22:03.046936 9236 tablet_bootstrap.cc:654] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.046936 9235 tablet_bootstrap.cc:654] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.047205 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:43519 (local address 127.0.0.1:56688)
0925 12:22:03.037027 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.037038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.037093 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.037265 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.037266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.037268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.037299 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.037300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.039598 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.039598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.040707 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.040709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.046958 (+ 6249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.046960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.047032 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.047038 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.047071 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.047123 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":78,"threads_started":1}
I20260925 12:22:03.047271 9235 log.cc:824] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:03.047271 9236 log.cc:824] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:03.047693 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.036518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.1.47.131:43519)
0925 12:22:03.036699 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.036704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.037108 (+ 404us) server_negotiation.cc:407] Connection header received
0925 12:22:03.037165 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.037167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.037182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.037257 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.037357 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.038440 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.040795 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.046830 (+ 6035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.047384 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.047490 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.047524 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.047575 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260925 12:22:03.048273 9235 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: No bootstrap required, opened a new log
I20260925 12:22:03.048364 9235 ts_tablet_manager.cc:1397] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:03.048579 9236 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: No bootstrap required, opened a new log
I20260925 12:22:03.048489 9141 tablet_service.cc:1467] Processing CreateTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e (DEFAULT_TABLE table=test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.048653 9236 ts_tablet_manager.cc:1397] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 12:22:03.048849 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec0fa4f7dbe84e949570e58a505ab67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.050652 9235 raft_consensus.cc:348] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.050859 9235 raft_consensus.cc:374] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.050899 9235 raft_consensus.cc:729] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.050820 9236 raft_consensus.cc:348] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.050968 9236 raft_consensus.cc:374] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.051002 9236 raft_consensus.cc:729] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.051061 9235 consensus_queue.cc:260] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.051110 9236 consensus_queue.cc:260] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.051386 9236 ts_tablet_manager.cc:1428] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:22:03.051386 9235 ts_tablet_manager.cc:1428] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:22:03.051414 9067 heartbeater.cc:503] Master 127.1.47.190:43525 was elected leader, sending a full tablet report...
I20260925 12:22:03.051534 8927 heartbeater.cc:503] Master 127.1.47.190:43525 was elected leader, sending a full tablet report...
I20260925 12:22:03.052984 9241 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Bootstrap starting.
I20260925 12:22:03.053578 9241 tablet_bootstrap.cc:654] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.053929 9241 log.cc:824] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:03.054641 9241 tablet_bootstrap.cc:492] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: No bootstrap required, opened a new log
I20260925 12:22:03.054734 9241 ts_tablet_manager.cc:1397] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:03.056438 9241 raft_consensus.cc:348] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.056607 9241 raft_consensus.cc:374] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.056659 9241 raft_consensus.cc:729] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.056833 9241 consensus_queue.cc:260] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.057129 9241 ts_tablet_manager.cc:1428] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 12:22:03.057281 9207 heartbeater.cc:503] Master 127.1.47.190:43525 was elected leader, sending a full tablet report...
I20260925 12:22:03.081542 9240 raft_consensus.cc:482] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:03.081678 9240 raft_consensus.cc:504] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.082018 9240 leader_election.cc:288] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:03.085299 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.082566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41819 (local address 127.1.47.131:43519)
0925 12:22:03.082613 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.082615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.082624 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:03.082653 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.082654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.082664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.082739 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.082896 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.083354 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.084416 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.084906 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.084981 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.084992 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.085189 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.085227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:22:03.085315 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.082333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:43519 (local address 127.1.47.129:41819)
0925 12:22:03.082546 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.082557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.082603 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.082742 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.082743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.082744 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.082820 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.082821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.083451 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.083452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.084313 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.084315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.085008 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.085009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.085092 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.085096 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.085182 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.085222 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":167,"threads_started":1}
I20260925 12:22:03.085449 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.082205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.1.47.129:41765)
0925 12:22:03.082269 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.082282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.082363 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.082588 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.082589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.082591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.082667 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.082670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.083416 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.083417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.084327 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.084329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.085173 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.085174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.085251 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.085253 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.085329 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.085373 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":13}
I20260925 12:22:03.085448 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.082302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41765 (local address 127.1.47.130:41101)
0925 12:22:03.082408 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.082410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.082423 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:22:03.082457 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.082458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.082472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.082578 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.082760 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.083300 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.084426 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.085058 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.085129 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.085138 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.085336 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.085373 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 12:22:03.085682 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0fa4f7dbe84e949570e58a505ab67e" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" is_pre_election: true
I20260925 12:22:03.085842 9161 raft_consensus.cc:2454] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 0.
I20260925 12:22:03.085861 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0fa4f7dbe84e949570e58a505ab67e" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6" is_pre_election: true
I20260925 12:22:03.086046 9021 raft_consensus.cc:2454] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 0.
I20260925 12:22:03.086063 8814 leader_election.cc:302] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30d732981c4bf5ad44d6ab7da6fbe9, 20001bb2a8ef48cc9c4c98ada978d0d4; no voters:
I20260925 12:22:03.086222 9240 raft_consensus.cc:2781] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:03.086294 9240 raft_consensus.cc:482] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:03.086336 9240 raft_consensus.cc:3037] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.087364 9240 raft_consensus.cc:504] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
W20260925 12:22:03.087417 9068 tablet.cc:2369] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:22:03.087517 9240 leader_election.cc:288] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 election: Requested vote from peers 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:03.087769 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0fa4f7dbe84e949570e58a505ab67e" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6"
I20260925 12:22:03.087787 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0fa4f7dbe84e949570e58a505ab67e" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
I20260925 12:22:03.087917 9021 raft_consensus.cc:3037] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.087922 9161 raft_consensus.cc:3037] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.088672 9021 raft_consensus.cc:2454] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 1.
I20260925 12:22:03.088778 9161 raft_consensus.cc:2454] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 1.
I20260925 12:22:03.088865 8814 leader_election.cc:302] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30d732981c4bf5ad44d6ab7da6fbe9, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.088958 9240 raft_consensus.cc:2781] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:03.089116 9240 raft_consensus.cc:686] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 LEADER]: Becoming Leader. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Running, Role: LEADER
I20260925 12:22:03.089257 9240 consensus_queue.cc:237] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } }
I20260925 12:22:03.090241 8724 catalog_manager.cc:5495] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } health_report { overall_health: HEALTHY } } }
I20260925 12:22:03.101990 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.099418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.1.47.129:34863)
0925 12:22:03.099491 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.099493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.099502 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:03.099547 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.099548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.099569 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.099649 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:03.099814 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.100346 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.101151 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.101352 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.101447 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.101637 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.101751 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.101837 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.101880 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.101900 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 12:22:03.103932 9161 raft_consensus.cc:1261] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Refusing update from remote peer 0a30d732981c4bf5ad44d6ab7da6fbe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.103932 9021 raft_consensus.cc:1261] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Refusing update from remote peer 0a30d732981c4bf5ad44d6ab7da6fbe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.104354 9245 consensus_queue.cc:1035] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.104503 9240 consensus_queue.cc:1035] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.107221 9248 mvcc.cc:205] Tried to move back new op lower bound from 7333228229030539264 to 7333228228974493696. Current Snapshot: MvccSnapshot[applied={T|T < 7333228229030539264}]
I20260925 12:22:03.107430 9249 mvcc.cc:205] Tried to move back new op lower bound from 7333228229030539264 to 7333228228974493696. Current Snapshot: MvccSnapshot[applied={T|T < 7333228229030539264}]
I20260925 12:22:03.107695 9250 mvcc.cc:205] Tried to move back new op lower bound from 7333228229030539264 to 7333228228974493696. Current Snapshot: MvccSnapshot[applied={T|T < 7333228229030539264 or (T in {7333228229030539264})}]
I20260925 12:22:03.109620 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:03.109720 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:03.110497 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Sending DeleteTablet for 3 replicas of tablet ec0fa4f7dbe84e949570e58a505ab67e
I20260925 12:22:03.110806 9000 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:55304
I20260925 12:22:03.110848 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:44576
I20260925 12:22:03.110860 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0fa4f7dbe84e949570e58a505ab67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56688
I20260925 12:22:03.111053 9259 tablet_replica.cc:330] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: stopping tablet replica
I20260925 12:22:03.111159 9259 raft_consensus.cc:2229] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:03.111339 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:22:03.111508 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:03.111558 9259 raft_consensus.cc:2258] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.112133 9259 ts_tablet_manager.cc:1905] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.112849 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 7350a2dd46fb464bbbc11039589caa6a (DEFAULT_TABLE table=test-table [id=367697f4093146ae96564fd6949d0af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.112857 9000 tablet_service.cc:1467] Processing CreateTablet for tablet 7350a2dd46fb464bbbc11039589caa6a (DEFAULT_TABLE table=test-table [id=367697f4093146ae96564fd6949d0af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.112885 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 7350a2dd46fb464bbbc11039589caa6a (DEFAULT_TABLE table=test-table [id=367697f4093146ae96564fd6949d0af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.112972 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7350a2dd46fb464bbbc11039589caa6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.112973 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7350a2dd46fb464bbbc11039589caa6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.112973 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7350a2dd46fb464bbbc11039589caa6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.114264 9235 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Bootstrap starting.
I20260925 12:22:03.114341 9236 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Bootstrap starting.
I20260925 12:22:03.114756 9235 tablet_bootstrap.cc:654] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.114755 9236 tablet_bootstrap.cc:654] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.115123 9241 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Bootstrap starting.
I20260925 12:22:03.115550 9241 tablet_bootstrap.cc:654] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.115659 9236 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: No bootstrap required, opened a new log
I20260925 12:22:03.115717 9236 ts_tablet_manager.cc:1397] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:03.115933 9236 raft_consensus.cc:348] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.116071 9236 raft_consensus.cc:374] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.116104 9236 raft_consensus.cc:729] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.116161 9235 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: No bootstrap required, opened a new log
I20260925 12:22:03.116149 9236 consensus_queue.cc:260] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.116209 9235 ts_tablet_manager.cc:1397] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:22:03.116250 9236 ts_tablet_manager.cc:1428] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:03.116376 9235 raft_consensus.cc:348] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.116453 9235 raft_consensus.cc:374] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.116479 9235 raft_consensus.cc:729] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.116526 9235 consensus_queue.cc:260] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.116614 9235 ts_tablet_manager.cc:1428] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:03.116894 9241 tablet_bootstrap.cc:492] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: No bootstrap required, opened a new log
I20260925 12:22:03.116943 9241 ts_tablet_manager.cc:1397] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:22:03.117076 9241 raft_consensus.cc:348] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.117164 9241 raft_consensus.cc:374] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.117195 9241 raft_consensus.cc:729] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.117244 9241 consensus_queue.cc:260] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.117360 9241 ts_tablet_manager.cc:1428] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 12:22:03.117496 9258 tablet_replica.cc:330] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: stopping tablet replica
I20260925 12:22:03.117811 9258 raft_consensus.cc:2229] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:03.117928 9260 tablet_replica.cc:330] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: stopping tablet replica
I20260925 12:22:03.117969 9258 raft_consensus.cc:2258] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.118024 9260 raft_consensus.cc:2229] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:22:03.118124 9260 raft_consensus.cc:2258] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.118448 9258 ts_tablet_manager.cc:1905] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.119249 9259 ts_tablet_manager.cc:1918] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:03.119349 9259 log.cc:1196] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/ec0fa4f7dbe84e949570e58a505ab67e
I20260925 12:22:03.119621 9259 ts_tablet_manager.cc:1939] T ec0fa4f7dbe84e949570e58a505ab67e P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting consensus metadata
I20260925 12:22:03.120064 8710 catalog_manager.cc:4841] TS 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519): tablet ec0fa4f7dbe84e949570e58a505ab67e (table test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]) successfully deleted
I20260925 12:22:03.120280 9260 ts_tablet_manager.cc:1905] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.120394 9258 ts_tablet_manager.cc:1918] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:03.120455 9258 log.cc:1196] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/ec0fa4f7dbe84e949570e58a505ab67e
I20260925 12:22:03.120632 9258 ts_tablet_manager.cc:1939] T ec0fa4f7dbe84e949570e58a505ab67e P 4f3a48986ba0476986acf81f930812f6: Deleting consensus metadata
I20260925 12:22:03.121150 8710 catalog_manager.cc:4841] TS 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101): tablet ec0fa4f7dbe84e949570e58a505ab67e (table test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]) successfully deleted
I20260925 12:22:03.122094 9260 ts_tablet_manager.cc:1918] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:03.122169 9260 log.cc:1196] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/ec0fa4f7dbe84e949570e58a505ab67e
I20260925 12:22:03.122421 9260 ts_tablet_manager.cc:1939] T ec0fa4f7dbe84e949570e58a505ab67e P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting consensus metadata
I20260925 12:22:03.122876 8708 catalog_manager.cc:4841] TS 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863): tablet ec0fa4f7dbe84e949570e58a505ab67e (table test-table [id=2dae81b0e5644cb187dd1bf1a2a624a7]) successfully deleted
W20260925 12:22:03.167850 8928 tablet.cc:2369] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:22:03.243333 9208 tablet.cc:2369] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:22:03.448001 9243 raft_consensus.cc:482] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:03.448118 9243 raft_consensus.cc:504] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.448549 9243 leader_election.cc:288] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863), 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101)
I20260925 12:22:03.452230 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.448891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:55611 (local address 127.1.47.129:34863)
0925 12:22:03.448940 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.448942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.448952 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:03.448987 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.448988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.448998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.449100 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.449411 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.450071 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451414 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.451772 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451834 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.451842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.452102 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.452154 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:22:03.452243 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.448785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34863 (local address 127.1.47.131:55611)
0925 12:22:03.448846 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.448858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.448905 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.449142 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.449143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.449145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.449242 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.449244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.450207 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.450208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.451277 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.451280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.451901 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.451992 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.451997 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.452092 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.452149 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":10}
I20260925 12:22:03.452267 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.448891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:50123 (local address 127.1.47.130:41101)
0925 12:22:03.448938 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.448941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.448979 (+ 38us) server_negotiation.cc:407] Connection header received
0925 12:22:03.449057 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.449058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.449067 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.449124 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.449380 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.449872 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451390 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.451793 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451952 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.451961 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.452152 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.452200 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:22:03.452581 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.448785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.1.47.131:50123)
0925 12:22:03.448922 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.448931 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.448988 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.449142 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.449143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.449145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.449242 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.449244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.449975 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.449976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.451277 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.451280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.451969 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.451970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.452047 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.452052 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.452136 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.452176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":28,"thread_start_us":59,"threads_started":1}
I20260925 12:22:03.452622 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7350a2dd46fb464bbbc11039589caa6a" candidate_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" is_pre_election: true
I20260925 12:22:03.452775 8881 raft_consensus.cc:2454] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20001bb2a8ef48cc9c4c98ada978d0d4 in term 0.
I20260925 12:22:03.452950 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7350a2dd46fb464bbbc11039589caa6a" candidate_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6" is_pre_election: true
I20260925 12:22:03.453040 9021 raft_consensus.cc:2454] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20001bb2a8ef48cc9c4c98ada978d0d4 in term 0.
I20260925 12:22:03.453025 9092 leader_election.cc:302] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30d732981c4bf5ad44d6ab7da6fbe9, 20001bb2a8ef48cc9c4c98ada978d0d4; no voters:
I20260925 12:22:03.453171 9243 raft_consensus.cc:2781] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:03.453239 9243 raft_consensus.cc:482] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:03.453279 9243 raft_consensus.cc:3037] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.453979 9243 raft_consensus.cc:504] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.454128 9243 leader_election.cc:288] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863), 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101)
I20260925 12:22:03.454275 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7350a2dd46fb464bbbc11039589caa6a" candidate_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6"
I20260925 12:22:03.454351 9021 raft_consensus.cc:3037] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.454314 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7350a2dd46fb464bbbc11039589caa6a" candidate_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
I20260925 12:22:03.454404 8881 raft_consensus.cc:3037] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.455019 9021 raft_consensus.cc:2454] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20001bb2a8ef48cc9c4c98ada978d0d4 in term 1.
I20260925 12:22:03.455135 8881 raft_consensus.cc:2454] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20001bb2a8ef48cc9c4c98ada978d0d4 in term 1.
I20260925 12:22:03.455190 9094 leader_election.cc:302] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20001bb2a8ef48cc9c4c98ada978d0d4, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.455291 9243 raft_consensus.cc:2781] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:03.455436 9243 raft_consensus.cc:686] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Running, Role: LEADER
I20260925 12:22:03.455497 9243 consensus_queue.cc:237] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.456363 8723 catalog_manager.cc:5495] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } health_report { overall_health: HEALTHY } } }
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.190@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:03.492547 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.484824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.1.47.190:43525)
0925 12:22:03.484887 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.484890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.485016 (+ 126us) server_negotiation.cc:407] Connection header received
0925 12:22:03.485046 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.485047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.485057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.485119 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:03.485705 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.486209 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.487099 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.487273 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.487322 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.489875 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:03.489876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:03.489876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:03.489883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:03.491295 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.491711 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.491712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.491713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.491767 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.491985 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.491986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.491987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.492100 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:03.492174 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.492328 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.492438 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 12:22:03.492617 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.484701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43525 (local address 127.0.0.1:34956)
0925 12:22:03.484879 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.484891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.484963 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.485180 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.485181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.485537 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:03.485591 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.485593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.486311 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.486312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.487003 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.487006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.487356 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.487358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.487420 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.488090 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:03.488097 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:03.489779 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:03.491400 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:03.491408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:03.491410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:03.491620 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:03.491832 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:03.491833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:03.491833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:03.491911 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:03.492165 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:03.492173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:03.492220 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.492306 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.492492 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":83,"thread_start_us":104,"threads_started":1}
I20260925 12:22:03.494710 8723 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:34956:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 12:22:03.494913 8723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:03.496479 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 89f32880946d4e44a5c5064b45229261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52ed4fb799574334b094380b72037935]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:22:03.496501 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 89f32880946d4e44a5c5064b45229261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52ed4fb799574334b094380b72037935]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:22:03.496642 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f32880946d4e44a5c5064b45229261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.496665 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f32880946d4e44a5c5064b45229261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.497951 9241 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: Bootstrap starting.
I20260925 12:22:03.498063 9000 tablet_service.cc:1467] Processing CreateTablet for tablet 89f32880946d4e44a5c5064b45229261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52ed4fb799574334b094380b72037935]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 12:22:03.498201 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f32880946d4e44a5c5064b45229261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.498378 9241 tablet_bootstrap.cc:654] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.499347 9241 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: No bootstrap required, opened a new log
I20260925 12:22:03.499402 9241 ts_tablet_manager.cc:1397] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:03.499593 9241 raft_consensus.cc:348] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.499670 9241 raft_consensus.cc:374] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.499688 9241 raft_consensus.cc:729] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.499730 9241 consensus_queue.cc:260] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.499831 9241 ts_tablet_manager.cc:1428] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:03.499933 9236 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Bootstrap starting.
I20260925 12:22:03.499920 9243 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.500039 9243 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.500298 9236 tablet_bootstrap.cc:654] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.500604 9235 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: Bootstrap starting.
I20260925 12:22:03.500967 9235 tablet_bootstrap.cc:654] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.501356 9236 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: No bootstrap required, opened a new log
I20260925 12:22:03.501399 9236 ts_tablet_manager.cc:1397] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:03.501528 9236 raft_consensus.cc:348] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.501610 9236 raft_consensus.cc:374] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.501611 9235 tablet_bootstrap.cc:492] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: No bootstrap required, opened a new log
I20260925 12:22:03.501634 9236 raft_consensus.cc:729] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.501648 9235 ts_tablet_manager.cc:1397] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:03.501678 9236 consensus_queue.cc:260] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.501781 9236 ts_tablet_manager.cc:1428] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:03.501798 9235 raft_consensus.cc:348] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.501873 9235 raft_consensus.cc:374] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.501900 9235 raft_consensus.cc:729] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.501907 9253 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.501950 9235 consensus_queue.cc:260] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.501993 9253 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.502044 9235 ts_tablet_manager.cc:1428] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:03.502110 9239 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.502221 9239 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.606650 9239 raft_consensus.cc:482] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:03.606801 9239 raft_consensus.cc:504] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.607170 9239 leader_election.cc:288] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519), 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863)
I20260925 12:22:03.610482 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.607518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:59253 (local address 127.1.47.129:34863)
0925 12:22:03.607573 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.607575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.607632 (+ 57us) server_negotiation.cc:407] Connection header received
0925 12:22:03.607772 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.607773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.607788 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.607842 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.608039 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.608678 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.609779 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.610131 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.610189 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.610195 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.610362 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.610404 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:22:03.610509 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.607500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:60479 (local address 127.1.47.131:43519)
0925 12:22:03.607662 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.607664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.607673 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:03.607704 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.607705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.607717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.607814 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:03.608101 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.608701 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.609746 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.610150 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.610246 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.610255 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.610402 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.610441 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260925 12:22:03.610515 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.607383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34863 (local address 127.1.47.130:59253)
0925 12:22:03.607577 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.607585 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.607656 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.607889 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.607890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.607893 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.607963 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.607964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.608764 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.608766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.609658 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.609660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.610212 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.610213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.610285 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.610289 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.610339 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.610405 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":33,"thread_start_us":61,"threads_started":1}
I20260925 12:22:03.610523 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.607383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:43519 (local address 127.1.47.130:60479)
0925 12:22:03.607527 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.607539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.607603 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.607822 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.607822 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.607825 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:03.607989 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.607991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.608814 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.608815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.609650 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.609652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.610260 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.610261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.610309 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.610312 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.610384 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.610430 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":30,"thread_start_us":64,"threads_started":1}
I20260925 12:22:03.611024 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f32880946d4e44a5c5064b45229261" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" is_pre_election: true
I20260925 12:22:03.611129 9161 raft_consensus.cc:2454] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 0.
I20260925 12:22:03.611090 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f32880946d4e44a5c5064b45229261" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" is_pre_election: true
I20260925 12:22:03.611205 8881 raft_consensus.cc:2454] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 0.
I20260925 12:22:03.611304 8954 leader_election.cc:302] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20001bb2a8ef48cc9c4c98ada978d0d4, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.611455 9239 raft_consensus.cc:2781] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:03.611497 9239 raft_consensus.cc:482] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:03.611518 9239 raft_consensus.cc:3037] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.612262 9239 raft_consensus.cc:504] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.612408 9239 leader_election.cc:288] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 election: Requested vote from peers 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519), 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863)
I20260925 12:22:03.612571 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f32880946d4e44a5c5064b45229261" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
I20260925 12:22:03.612565 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f32880946d4e44a5c5064b45229261" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
I20260925 12:22:03.612643 9161 raft_consensus.cc:3037] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.612643 8881 raft_consensus.cc:3037] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.613426 8881 raft_consensus.cc:2454] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 1.
I20260925 12:22:03.613435 9161 raft_consensus.cc:2454] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 1.
I20260925 12:22:03.613700 8954 leader_election.cc:302] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20001bb2a8ef48cc9c4c98ada978d0d4, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.613786 9239 raft_consensus.cc:2781] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:03.613901 9239 raft_consensus.cc:686] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Running, Role: LEADER
I20260925 12:22:03.613992 9239 consensus_queue.cc:237] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } }
I20260925 12:22:03.614355 9279 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: TxnStatusTablet state changed. Reason: New leader 4f3a48986ba0476986acf81f930812f6. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.614457 9279 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:22:03.614590 9281 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 12:22:03.614713 9281 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 12:22:03.614781 8723 catalog_manager.cc:5495] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f3a48986ba0476986acf81f930812f6 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } health_report { overall_health: HEALTHY } } }
I20260925 12:22:03.623387 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.620214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.0.0.1:55310)
0925 12:22:03.620292 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.620301 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.620335 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.620500 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.620501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.620502 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:22:03.620546 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.620548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.621427 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.621428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.622235 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.622238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.622742 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.622743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.622793 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.622810 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.623193 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.623203 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.623263 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.623317 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 12:22:03.623366 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.620284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.1.47.130:41101)
0925 12:22:03.620327 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.620329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.620339 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:03.620400 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.620401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.620420 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.620502 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:03.620585 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.621304 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.622325 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.622633 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.622711 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.622998 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.623118 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.623209 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.623268 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.623290 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:22:03.624603 8881 raft_consensus.cc:1261] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Refusing update from remote peer 4f3a48986ba0476986acf81f930812f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.624816 9161 raft_consensus.cc:1261] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Refusing update from remote peer 4f3a48986ba0476986acf81f930812f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.625092 9279 consensus_queue.cc:1035] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.625278 9239 consensus_queue.cc:1035] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.625810 9243 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: TxnStatusTablet state changed. Reason: New leader 4f3a48986ba0476986acf81f930812f6. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.625909 9243 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.626359 9253 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: TxnStatusTablet state changed. Reason: New leader 4f3a48986ba0476986acf81f930812f6. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.626470 9253 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.627018 9279 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.627104 9279 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:22:03.627156 9279 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.627192 9279 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 4f3a48986ba0476986acf81f930812f6: This TxnStatusTablet replica's current role is: LEADER
I20260925 12:22:03.627203 9243 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.627290 9243 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 20001bb2a8ef48cc9c4c98ada978d0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.628049 9253 tablet_replica.cc:439] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } } peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } }
I20260925 12:22:03.628132 9253 tablet_replica.cc:442] T 89f32880946d4e44a5c5064b45229261 P 0a30d732981c4bf5ad44d6ab7da6fbe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 12:22:03.631541 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.628862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.1.47.131:43519)
0925 12:22:03.628925 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.628926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.628934 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:03.629027 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.629028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.629049 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.629132 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:03.629245 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.629990 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.630700 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.630996 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.631067 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.631196 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.631324 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.631409 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.631425 (+ 16us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.631446 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 12:22:03.636860 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.634040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.1.47.131:37745)
0925 12:22:03.634212 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.634227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.634278 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.634414 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.634415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.634430 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:22:03.634487 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.634488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.635357 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.635358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.635996 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.635998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.636503 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.636504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.636551 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.636563 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.636731 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.636732 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.636765 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.636799 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":75,"threads_started":1}
I20260925 12:22:03.636863 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.634125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:37745 (local address 127.1.47.130:41101)
0925 12:22:03.634168 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.634170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.634315 (+ 145us) server_negotiation.cc:407] Connection header received
0925 12:22:03.634342 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.634343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.634354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.634412 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:03.634546 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.635286 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.636080 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.636387 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.636502 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.636640 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.636673 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.636725 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.636768 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.636791 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:22:03.639477 9021 raft_consensus.cc:1261] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Refusing update from remote peer 20001bb2a8ef48cc9c4c98ada978d0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.639499 8881 raft_consensus.cc:1261] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Refusing update from remote peer 20001bb2a8ef48cc9c4c98ada978d0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:03.639791 9265 consensus_queue.cc:1035] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.640013 9265 consensus_queue.cc:1035] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:03.649232 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.646277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:54485 (local address 127.1.47.131:43519)
0925 12:22:03.646319 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.646321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.646431 (+ 110us) server_negotiation.cc:407] Connection header received
0925 12:22:03.646467 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.646468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.646478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.646544 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:03.646665 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.647431 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.648489 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.648790 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.648864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.649003 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.649037 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.649089 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.649133 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.649154 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:22:03.649318 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.646220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:43519 (local address 127.1.47.130:54485)
0925 12:22:03.646354 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:03.646365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:03.646431 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:03.646546 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:03.646546 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:03.646547 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:22:03.646616 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.646617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.647571 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.647572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.648405 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:03.648407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.648865 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.648866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:03.648911 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.648927 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:22:03.649095 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:22:03.649097 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:22:03.649129 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 12:22:03.649198 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260925 12:22:03.658103 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:03.658263 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:03.658914 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Sending DeleteTablet for 3 replicas of tablet 7350a2dd46fb464bbbc11039589caa6a
I20260925 12:22:03.659164 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 7350a2dd46fb464bbbc11039589caa6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:44576
I20260925 12:22:03.659192 9000 tablet_service.cc:1514] Processing DeleteTablet for tablet 7350a2dd46fb464bbbc11039589caa6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:55304
I20260925 12:22:03.659395 9305 tablet_replica.cc:330] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: stopping tablet replica
I20260925 12:22:03.659480 9305 raft_consensus.cc:2229] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:03.659519 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 7350a2dd46fb464bbbc11039589caa6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:03 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56688
I20260925 12:22:03.659543 9305 raft_consensus.cc:2258] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.659825 9305 ts_tablet_manager.cc:1905] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.661757 9305 ts_tablet_manager.cc:1918] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:22:03.661901 9305 log.cc:1196] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/7350a2dd46fb464bbbc11039589caa6a
I20260925 12:22:03.662233 9305 ts_tablet_manager.cc:1939] T 7350a2dd46fb464bbbc11039589caa6a P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting consensus metadata
I20260925 12:22:03.662644 8708 catalog_manager.cc:4841] TS 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863): tablet 7350a2dd46fb464bbbc11039589caa6a (table test-table [id=367697f4093146ae96564fd6949d0af5]) successfully deleted
I20260925 12:22:03.663925 9307 tablet_replica.cc:330] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: stopping tablet replica
I20260925 12:22:03.663995 9307 raft_consensus.cc:2229] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:22:03.664085 9307 raft_consensus.cc:2258] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.664338 9307 ts_tablet_manager.cc:1905] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.665947 9307 ts_tablet_manager.cc:1918] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:22:03.666013 9307 log.cc:1196] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/7350a2dd46fb464bbbc11039589caa6a
I20260925 12:22:03.666234 9307 ts_tablet_manager.cc:1939] T 7350a2dd46fb464bbbc11039589caa6a P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting consensus metadata
I20260925 12:22:03.666415 9306 tablet_replica.cc:330] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: stopping tablet replica
I20260925 12:22:03.666487 9306 raft_consensus.cc:2229] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:03.666601 9306 raft_consensus.cc:2258] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:03.666843 9306 ts_tablet_manager.cc:1905] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:03.666829 8710 catalog_manager.cc:4841] TS 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519): tablet 7350a2dd46fb464bbbc11039589caa6a (table test-table [id=367697f4093146ae96564fd6949d0af5]) successfully deleted
I20260925 12:22:03.668463 9306 ts_tablet_manager.cc:1918] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:22:03.668527 9306 log.cc:1196] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/7350a2dd46fb464bbbc11039589caa6a
I20260925 12:22:03.668761 9306 ts_tablet_manager.cc:1939] T 7350a2dd46fb464bbbc11039589caa6a P 4f3a48986ba0476986acf81f930812f6: Deleting consensus metadata
I20260925 12:22:03.669063 8710 catalog_manager.cc:4841] TS 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101): tablet 7350a2dd46fb464bbbc11039589caa6a (table test-table [id=367697f4093146ae96564fd6949d0af5]) successfully deleted
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 12:22:03.699276 8786 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:03.691799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.1.47.190:43525)
0925 12:22:03.691878 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.691880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.692025 (+ 145us) server_negotiation.cc:407] Connection header received
0925 12:22:03.692072 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.692073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.692085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.692169 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:03.692897 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.693783 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.694911 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.695258 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.695375 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.699207 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34958: BlockingRecv error: recv got EOF from 127.0.0.1:34958 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 25 12:22:03 dist-test-slave-l9ww krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM
I20260925 12:22:03.709753 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:03.700237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.1.47.190:43525)
0925 12:22:03.700272 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:03.700273 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:03.700510 (+ 237us) server_negotiation.cc:407] Connection header received
0925 12:22:03.700577 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:03.700578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:03.700588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:03.700645 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:03.701276 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.702044 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.703194 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:03.703481 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:03.703557 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:03.706461 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:03.706463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:03.706463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:03.706472 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:03.708387 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.708814 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.708816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.708816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.708884 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:03.709122 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:03.709123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:03.709123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:03.709282 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:03.709397 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:03.709550 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 12:22:03.709640 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:22:03.711810 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:22:03.712025 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:03.713367 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 1416331440084d65b1398629edc23dca (DEFAULT_TABLE table=test-table [id=9e8cc815d942448caf5c82f2a2895527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.713430 9000 tablet_service.cc:1467] Processing CreateTablet for tablet 1416331440084d65b1398629edc23dca (DEFAULT_TABLE table=test-table [id=9e8cc815d942448caf5c82f2a2895527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.713514 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1416331440084d65b1398629edc23dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.713567 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1416331440084d65b1398629edc23dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.714900 9236 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Bootstrap starting.
I20260925 12:22:03.715094 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 1416331440084d65b1398629edc23dca (DEFAULT_TABLE table=test-table [id=9e8cc815d942448caf5c82f2a2895527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:03.715227 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1416331440084d65b1398629edc23dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:03.715252 9236 tablet_bootstrap.cc:654] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.715960 9235 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Bootstrap starting.
I20260925 12:22:03.716383 9235 tablet_bootstrap.cc:654] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.716655 9241 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Bootstrap starting.
I20260925 12:22:03.717103 9241 tablet_bootstrap.cc:654] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:03.717361 9235 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: No bootstrap required, opened a new log
I20260925 12:22:03.717422 9235 ts_tablet_manager.cc:1397] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:03.717573 9235 raft_consensus.cc:348] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.717644 9235 raft_consensus.cc:374] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.717665 9235 raft_consensus.cc:729] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.717705 9235 consensus_queue.cc:260] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.717794 9235 ts_tablet_manager.cc:1428] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:03.718133 9241 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: No bootstrap required, opened a new log
I20260925 12:22:03.718180 9241 ts_tablet_manager.cc:1397] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:22:03.718345 9241 raft_consensus.cc:348] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.718428 9241 raft_consensus.cc:374] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.718451 9241 raft_consensus.cc:729] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.718495 9241 consensus_queue.cc:260] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.718591 9241 ts_tablet_manager.cc:1428] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 12:22:03.719040 9236 tablet_bootstrap.cc:492] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: No bootstrap required, opened a new log
I20260925 12:22:03.719095 9236 ts_tablet_manager.cc:1397] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 12:22:03.719208 9236 raft_consensus.cc:348] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.719271 9236 raft_consensus.cc:374] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:03.719290 9236 raft_consensus.cc:729] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Initialized, Role: FOLLOWER
I20260925 12:22:03.719331 9236 consensus_queue.cc:260] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.719416 9236 ts_tablet_manager.cc:1428] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:03.946650 9279 raft_consensus.cc:482] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:03.946763 9279 raft_consensus.cc:504] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.946978 9279 leader_election.cc:288] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:03.947232 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1416331440084d65b1398629edc23dca" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" is_pre_election: true
I20260925 12:22:03.947232 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1416331440084d65b1398629edc23dca" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" is_pre_election: true
I20260925 12:22:03.947340 8881 raft_consensus.cc:2454] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 0.
I20260925 12:22:03.947341 9161 raft_consensus.cc:2454] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 0.
I20260925 12:22:03.947512 8954 leader_election.cc:302] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20001bb2a8ef48cc9c4c98ada978d0d4, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.947623 9279 raft_consensus.cc:2781] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:03.947662 9279 raft_consensus.cc:482] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:03.947685 9279 raft_consensus.cc:3037] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.948455 9279 raft_consensus.cc:504] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.948585 9279 leader_election.cc:288] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 election: Requested vote from peers 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:03.948748 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1416331440084d65b1398629edc23dca" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
I20260925 12:22:03.948825 9161 raft_consensus.cc:3037] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.948746 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1416331440084d65b1398629edc23dca" candidate_uuid: "4f3a48986ba0476986acf81f930812f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9"
I20260925 12:22:03.948858 8881 raft_consensus.cc:3037] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:03.949496 9161 raft_consensus.cc:2454] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 1.
I20260925 12:22:03.949597 8881 raft_consensus.cc:2454] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3a48986ba0476986acf81f930812f6 in term 1.
I20260925 12:22:03.949679 8954 leader_election.cc:302] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20001bb2a8ef48cc9c4c98ada978d0d4, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:03.949783 9279 raft_consensus.cc:2781] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:03.949836 9279 raft_consensus.cc:686] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Running, Role: LEADER
I20260925 12:22:03.949887 9279 consensus_queue.cc:237] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:03.950446 8724 catalog_manager.cc:5495] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f3a48986ba0476986acf81f930812f6 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "4f3a48986ba0476986acf81f930812f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } health_report { overall_health: UNKNOWN } } }
I20260925 12:22:04.004654 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.002406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.1.47.130:41101)
0925 12:22:04.002498 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:04.002500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:04.002510 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:04.002534 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:04.002534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:04.002545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:04.002606 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:04.002753 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.003218 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.003959 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.004200 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.004266 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.004387 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.004418 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.004462 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:04.004509 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 12:22:04.004530 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 12:22:04.005636 9161 raft_consensus.cc:1261] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Refusing update from remote peer 4f3a48986ba0476986acf81f930812f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:04.005658 8881 raft_consensus.cc:1261] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Refusing update from remote peer 4f3a48986ba0476986acf81f930812f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:04.005870 9279 consensus_queue.cc:1035] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:04.006035 9239 consensus_queue.cc:1035] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:04.009130 8723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:04.009233 8723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:04.009891 8723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Sending DeleteTablet for 3 replicas of tablet 1416331440084d65b1398629edc23dca
I20260925 12:22:04.010135 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 1416331440084d65b1398629edc23dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:44576
I20260925 12:22:04.010142 9000 tablet_service.cc:1514] Processing DeleteTablet for tablet 1416331440084d65b1398629edc23dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:55304
I20260925 12:22:04.010176 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 1416331440084d65b1398629edc23dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56688
I20260925 12:22:04.010227 9305 tablet_replica.cc:330] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: stopping tablet replica
I20260925 12:22:04.010227 9306 tablet_replica.cc:330] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: stopping tablet replica
I20260925 12:22:04.010267 9307 tablet_replica.cc:330] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: stopping tablet replica
I20260925 12:22:04.010275 9306 raft_consensus.cc:2229] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:22:04.010277 9305 raft_consensus.cc:2229] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:04.010316 9307 raft_consensus.cc:2229] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:04.010330 9305 raft_consensus.cc:2258] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.010358 9307 raft_consensus.cc:2258] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.010360 9306 raft_consensus.cc:2258] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.010675 9307 ts_tablet_manager.cc:1905] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:04.011009 9306 ts_tablet_manager.cc:1905] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:04.012259 9306 ts_tablet_manager.cc:1918] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:04.012343 9306 log.cc:1196] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/1416331440084d65b1398629edc23dca
I20260925 12:22:04.012377 9307 ts_tablet_manager.cc:1918] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:04.012442 9307 log.cc:1196] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/1416331440084d65b1398629edc23dca
I20260925 12:22:04.012584 9306 ts_tablet_manager.cc:1939] T 1416331440084d65b1398629edc23dca P 4f3a48986ba0476986acf81f930812f6: Deleting consensus metadata
I20260925 12:22:04.012696 9307 ts_tablet_manager.cc:1939] T 1416331440084d65b1398629edc23dca P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting consensus metadata
I20260925 12:22:04.012900 8710 catalog_manager.cc:4841] TS 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101): tablet 1416331440084d65b1398629edc23dca (table test-table [id=9e8cc815d942448caf5c82f2a2895527]) successfully deleted
I20260925 12:22:04.013088 8710 catalog_manager.cc:4841] TS 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519): tablet 1416331440084d65b1398629edc23dca (table test-table [id=9e8cc815d942448caf5c82f2a2895527]) successfully deleted
I20260925 12:22:04.013172 8723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 12:22:04.013252 9305 ts_tablet_manager.cc:1905] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:22:04.013358 8723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:04.014541 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 54f7a2faf02c41ca9688382909e171d4 (DEFAULT_TABLE table=test-table [id=ef2000033302435f8268440bbaca7070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:04.014568 9000 tablet_service.cc:1467] Processing CreateTablet for tablet 54f7a2faf02c41ca9688382909e171d4 (DEFAULT_TABLE table=test-table [id=ef2000033302435f8268440bbaca7070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:04.014658 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f7a2faf02c41ca9688382909e171d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:04.014658 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f7a2faf02c41ca9688382909e171d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:04.014770 9305 ts_tablet_manager.cc:1918] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:04.014838 9305 log.cc:1196] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/1416331440084d65b1398629edc23dca
I20260925 12:22:04.015098 9305 ts_tablet_manager.cc:1939] T 1416331440084d65b1398629edc23dca P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting consensus metadata
I20260925 12:22:04.015938 9236 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Bootstrap starting.
I20260925 12:22:04.015954 9235 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Bootstrap starting.
I20260925 12:22:04.016268 9236 tablet_bootstrap.cc:654] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:04.016297 9235 tablet_bootstrap.cc:654] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:04.016566 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 54f7a2faf02c41ca9688382909e171d4 (DEFAULT_TABLE table=test-table [id=ef2000033302435f8268440bbaca7070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:04.016609 8708 catalog_manager.cc:4841] TS 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863): tablet 1416331440084d65b1398629edc23dca (table test-table [id=9e8cc815d942448caf5c82f2a2895527]) successfully deleted
I20260925 12:22:04.016669 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f7a2faf02c41ca9688382909e171d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:04.016835 9236 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: No bootstrap required, opened a new log
I20260925 12:22:04.016867 9236 ts_tablet_manager.cc:1397] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:04.016947 9236 raft_consensus.cc:348] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.016995 9236 raft_consensus.cc:374] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:04.017010 9236 raft_consensus.cc:729] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Initialized, Role: FOLLOWER
I20260925 12:22:04.017081 9236 consensus_queue.cc:260] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.017179 9236 ts_tablet_manager.cc:1428] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:04.017890 9241 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Bootstrap starting.
I20260925 12:22:04.018208 9235 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: No bootstrap required, opened a new log
I20260925 12:22:04.018224 9241 tablet_bootstrap.cc:654] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:04.018253 9235 ts_tablet_manager.cc:1397] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:22:04.018365 9235 raft_consensus.cc:348] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.018431 9235 raft_consensus.cc:374] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:04.018455 9235 raft_consensus.cc:729] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3a48986ba0476986acf81f930812f6, State: Initialized, Role: FOLLOWER
I20260925 12:22:04.018504 9235 consensus_queue.cc:260] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.018599 9235 ts_tablet_manager.cc:1428] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:04.018880 9241 tablet_bootstrap.cc:492] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: No bootstrap required, opened a new log
I20260925 12:22:04.018929 9241 ts_tablet_manager.cc:1397] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:04.019071 9241 raft_consensus.cc:348] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.019137 9241 raft_consensus.cc:374] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:04.019164 9241 raft_consensus.cc:729] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20001bb2a8ef48cc9c4c98ada978d0d4, State: Initialized, Role: FOLLOWER
I20260925 12:22:04.019210 9241 consensus_queue.cc:260] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.019315 9241 ts_tablet_manager.cc:1428] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:04.044229 9253 raft_consensus.cc:482] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:04.044335 9253 raft_consensus.cc:504] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.044490 9253 leader_election.cc:288] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:04.044731 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f7a2faf02c41ca9688382909e171d4" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" is_pre_election: true
I20260925 12:22:04.044725 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f7a2faf02c41ca9688382909e171d4" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6" is_pre_election: true
I20260925 12:22:04.044836 9161 raft_consensus.cc:2454] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 0.
I20260925 12:22:04.044839 9021 raft_consensus.cc:2454] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 0.
I20260925 12:22:04.045008 8814 leader_election.cc:302] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30d732981c4bf5ad44d6ab7da6fbe9, 4f3a48986ba0476986acf81f930812f6; no voters:
I20260925 12:22:04.045126 9253 raft_consensus.cc:2781] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:04.045193 9253 raft_consensus.cc:482] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:04.045226 9253 raft_consensus.cc:3037] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:04.045936 9253 raft_consensus.cc:504] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.046075 9253 leader_election.cc:288] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 election: Requested vote from peers 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101), 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519)
I20260925 12:22:04.046216 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f7a2faf02c41ca9688382909e171d4" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3a48986ba0476986acf81f930812f6"
I20260925 12:22:04.046298 9021 raft_consensus.cc:3037] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:04.046241 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f7a2faf02c41ca9688382909e171d4" candidate_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4"
I20260925 12:22:04.046336 9161 raft_consensus.cc:3037] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:04.046888 9161 raft_consensus.cc:2454] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 1.
I20260925 12:22:04.047053 9021 raft_consensus.cc:2454] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30d732981c4bf5ad44d6ab7da6fbe9 in term 1.
I20260925 12:22:04.047070 8814 leader_election.cc:302] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30d732981c4bf5ad44d6ab7da6fbe9, 20001bb2a8ef48cc9c4c98ada978d0d4; no voters:
I20260925 12:22:04.047170 9253 raft_consensus.cc:2781] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:04.047287 9253 raft_consensus.cc:686] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 LEADER]: Becoming Leader. State: Replica: 0a30d732981c4bf5ad44d6ab7da6fbe9, State: Running, Role: LEADER
I20260925 12:22:04.047349 9253 consensus_queue.cc:237] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } }
I20260925 12:22:04.048002 8723 catalog_manager.cc:5495] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a30d732981c4bf5ad44d6ab7da6fbe9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 } health_report { overall_health: UNKNOWN } } }
I20260925 12:22:04.058977 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.056496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.1.47.129:34863)
0925 12:22:04.056547 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:04.056550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:04.056560 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:04.056627 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:04.056628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:04.056639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:04.056692 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:04.056842 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.057371 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.058209 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.058446 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.058529 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.058687 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.058723 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.058794 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:04.058868 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 12:22:04.058887 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:22:04.064332 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.061612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41101 (local address 127.1.47.129:58905)
0925 12:22:04.061807 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:04.061817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:04.061871 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:04.061984 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:04.061985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:04.061986 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:22:04.062020 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:04.062021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.062868 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.062868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:04.063458 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:04.063460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.063952 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.063953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:04.064005 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.064020 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.064200 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.064202 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:22:04.064232 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 12:22:04.064270 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260925 12:22:04.064343 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.061719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:58905 (local address 127.1.47.130:41101)
0925 12:22:04.061771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:04.061773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:04.061879 (+ 106us) server_negotiation.cc:407] Connection header received
0925 12:22:04.061908 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:04.061909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:04.061920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:04.061984 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:04.062070 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.062787 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.063541 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.063843 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.063939 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.064107 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.064141 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.064195 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:04.064248 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 12:22:04.064267 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:22:04.066435 9021 raft_consensus.cc:1261] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Refusing update from remote peer 0a30d732981c4bf5ad44d6ab7da6fbe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:04.066470 9161 raft_consensus.cc:1261] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Refusing update from remote peer 0a30d732981c4bf5ad44d6ab7da6fbe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:04.066666 9331 consensus_queue.cc:1035] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3a48986ba0476986acf81f930812f6" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:04.066753 9253 consensus_queue.cc:1035] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20001bb2a8ef48cc9c4c98ada978d0d4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:04.073717 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.071286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:53359 (local address 127.1.47.129:34863)
0925 12:22:04.071325 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:04.071327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:04.071335 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:04.071390 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:04.071390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:04.071400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:04.071488 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:04.071601 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.072139 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.073108 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.073315 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.073367 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.073520 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.073545 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.073577 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:04.073643 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 12:22:04.073661 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:22:04.073764 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.071190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34863 (local address 127.1.47.130:53359)
0925 12:22:04.071289 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:04.071301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:04.071337 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:04.071500 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:04.071501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:04.071502 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 12:22:04.071546 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:04.071548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.072210 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.072211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:04.073031 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:04.073033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:04.073389 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:04.073390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:04.073446 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:04.073453 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 12:22:04.073593 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 12:22:04.073594 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 12:22:04.073641 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 12:22:04.073685 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 12:22:04.082455 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:04.082581 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:04.083233 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 912332e85a9c4313b2876ffd60bf9d45: Sending DeleteTablet for 3 replicas of tablet 54f7a2faf02c41ca9688382909e171d4
I20260925 12:22:04.083523 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f7a2faf02c41ca9688382909e171d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:44576
I20260925 12:22:04.083561 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f7a2faf02c41ca9688382909e171d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56688
I20260925 12:22:04.083664 9305 tablet_replica.cc:330] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: stopping tablet replica
I20260925 12:22:04.083726 9305 raft_consensus.cc:2229] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:22:04.083739 9000 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f7a2faf02c41ca9688382909e171d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:04 UTC) from {username='oryx', principal='oryx/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:55304
I20260925 12:22:04.083801 9305 raft_consensus.cc:2258] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.083844 9306 tablet_replica.cc:330] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: stopping tablet replica
I20260925 12:22:04.083889 9307 tablet_replica.cc:330] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: stopping tablet replica
I20260925 12:22:04.083921 9306 raft_consensus.cc:2229] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:04.083946 9307 raft_consensus.cc:2229] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:04.083966 9306 raft_consensus.cc:2258] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.083981 9307 raft_consensus.cc:2258] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:04.084260 9305 ts_tablet_manager.cc:1905] T 54f7a2faf02c41ca9688382909e171d4 P 0a30d732981c4bf5ad44d6ab7da6fbe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:04.084260 9306 ts_tablet_manager.cc:1905] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:04.084304 9307 ts_tablet_manager.cc:1905] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:04.085987 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8796
I20260925 12:22:04.086666 9307 ts_tablet_manager.cc:1918] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:22:04.086720 9307 log.cc:1196] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/54f7a2faf02c41ca9688382909e171d4
I20260925 12:22:04.086953 9307 ts_tablet_manager.cc:1939] T 54f7a2faf02c41ca9688382909e171d4 P 20001bb2a8ef48cc9c4c98ada978d0d4: Deleting consensus metadata
I20260925 12:22:04.087250 8710 catalog_manager.cc:4841] TS 20001bb2a8ef48cc9c4c98ada978d0d4 (127.1.47.131:43519): tablet 54f7a2faf02c41ca9688382909e171d4 (table test-table [id=ef2000033302435f8268440bbaca7070]) successfully deleted
I20260925 12:22:04.087319 9306 ts_tablet_manager.cc:1918] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 12:22:04.087383 9306 log.cc:1196] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/54f7a2faf02c41ca9688382909e171d4
I20260925 12:22:04.087589 9306 ts_tablet_manager.cc:1939] T 54f7a2faf02c41ca9688382909e171d4 P 4f3a48986ba0476986acf81f930812f6: Deleting consensus metadata
I20260925 12:22:04.087934 8710 catalog_manager.cc:4841] TS 4f3a48986ba0476986acf81f930812f6 (127.1.47.130:41101): tablet 54f7a2faf02c41ca9688382909e171d4 (table test-table [id=ef2000033302435f8268440bbaca7070]) successfully deleted
W20260925 12:22:04.093066 8708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:34863 (error 108)
I20260925 12:22:04.093298 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8936
I20260925 12:22:04.094995 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:04.094791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34863 (local address 127.0.0.1:44620)
0925 12:22:04.094885 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:04.094928 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:34863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":53,"threads_started":1}
W20260925 12:22:04.095120 8708 catalog_manager.cc:4584] TS 0a30d732981c4bf5ad44d6ab7da6fbe9 (127.1.47.129:34863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54f7a2faf02c41ca9688382909e171d4: Network error: Client connection negotiation failed: client connection to 127.1.47.129:34863: connect: Connection refused (error 111)
I20260925 12:22:04.099460 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9076
I20260925 12:22:04.105259 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 8692
2026-09-25T12:22:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2929 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:22:04 dist-test-slave-l9ww krb5kdc[9343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:22:04 dist-test-slave-l9ww krb5kdc[9343](info): set up 2 sockets
Sep 25 12:22:04 dist-test-slave-l9ww krb5kdc[9343](info): commencing operation
krb5kdc: starting...
W20260925 12:22:05.148818 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:22:05 dist-test-slave-l9ww krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:22:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:22:05.271559 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36047
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:05.351096 9356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:05.351358 9356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:05.351397 9356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:05.352609 9356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:05.352694 9356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:05.352718 9356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:05.352739 9356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:05.352768 9356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:05.354302 9356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:05.354588 9356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:05.354825 9356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:05.359979 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:05.359979 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:05.360010 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:05.360047 9356 server_base.cc:1034] running on GCE node
I20260925 12:22:05.360463 9356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:05.360757 9356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:05.361948 9356 hybrid_clock.cc:648] HybridClock initialized: now 1790338925361935 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T12:22:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1263 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:22:05 dist-test-slave-l9ww krb5kdc[9369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:22:05 dist-test-slave-l9ww krb5kdc[9369](info): set up 2 sockets
Sep 25 12:22:05 dist-test-slave-l9ww krb5kdc[9369](info): commencing operation
krb5kdc: starting...
W20260925 12:22:06.488584 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:22:06 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:22:06.615022 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46377
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:06.698710 9382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:06.698951 9382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:06.698974 9382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:06.700160 9382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:06.700212 9382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:06.700233 9382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:06.700248 9382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:06.700263 9382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:06.701700 9382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:06.701910 9382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:06.702099 9382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:06.707288 9390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:06.707274 9388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:06.707427 9382 server_base.cc:1034] running on GCE node
W20260925 12:22:06.707355 9387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:06.707847 9382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:06.708375 9382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:06.709545 9382 hybrid_clock.cc:648] HybridClock initialized: now 1790338926709520 us; error 44 us; skew 500 ppm
Sep 25 12:22:06 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:06.712082 9382 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:22:06.712570 9382 webserver.cc:460] Webserver started at http://127.1.47.190:38727/ using document root <none> and password file <none>
I20260925 12:22:06.712823 9382 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:06.712893 9382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:06.713042 9382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:06.714110 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "7d77cd4f53f84ad6b07e5e60b036673c"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "b11f76ed3cc36eefa0d8f93eb8df3fae"
server_key_iv: "36baebb01b69eaec7c3bef3d8a0c9ebc"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.714486 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "7d77cd4f53f84ad6b07e5e60b036673c"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "b11f76ed3cc36eefa0d8f93eb8df3fae"
server_key_iv: "36baebb01b69eaec7c3bef3d8a0c9ebc"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.715929 9382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 12:22:06.716734 9397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:06.716962 9382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:06.717073 9382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "7d77cd4f53f84ad6b07e5e60b036673c"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "b11f76ed3cc36eefa0d8f93eb8df3fae"
server_key_iv: "36baebb01b69eaec7c3bef3d8a0c9ebc"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.717161 9382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:06.737365 9382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:06.739768 9382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:06.739990 9382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:06.744539 9382 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46377
I20260925 12:22:06.744626 9449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46377 every 8 connection(s)
I20260925 12:22:06.745009 9382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:06.745952 9450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:06.748672 9450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Bootstrap starting.
I20260925 12:22:06.749330 9450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:06.749656 9450 log.cc:824] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:06.750357 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9382
I20260925 12:22:06.750478 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:22:06.750610 9450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: No bootstrap required, opened a new log
I20260925 12:22:06.750670 1214 external_mini_cluster.cc:1428] Setting key 9b355cc716e944c58af2d31492f51584
I20260925 12:22:06.752682 9450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:06.752847 9450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:06.752873 9450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d77cd4f53f84ad6b07e5e60b036673c, State: Initialized, Role: FOLLOWER
I20260925 12:22:06.753023 9450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:06.753113 9450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:06.753165 9450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:06.753238 9450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:06.754400 9450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:06.754504 9450 leader_election.cc:302] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d77cd4f53f84ad6b07e5e60b036673c; no voters:
I20260925 12:22:06.754716 9450 leader_election.cc:288] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:22:06.754778 9455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:06.755000 9455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 LEADER]: Becoming Leader. State: Replica: 7d77cd4f53f84ad6b07e5e60b036673c, State: Running, Role: LEADER
I20260925 12:22:06.755052 9450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:06.755152 9455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:06.755560 9457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d77cd4f53f84ad6b07e5e60b036673c. Latest consensus state: current_term: 1 leader_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } } }
I20260925 12:22:06.755766 9457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:06.755561 9456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } } }
I20260925 12:22:06.755929 9456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:06.756103 9463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:06.756703 9463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:06.758739 9463 catalog_manager.cc:1347] Generated new cluster ID: ae7b799ebc3c49c7899f7a32de27c3f9
I20260925 12:22:06.758800 9463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:22:06 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:06.764156 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:06.751266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.1.47.190:46377)
0925 12:22:06.751481 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:06.751496 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:06.751524 (+ 28us) server_negotiation.cc:407] Connection header received
0925 12:22:06.752689 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:06.752699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:06.752778 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:06.752973 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:06.753994 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.755439 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.756740 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.757375 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.757536 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:06.760308 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:06.760312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:06.760313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:06.760325 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:06.762218 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:06.762613 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:06.762616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:06.762617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:06.762736 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:06.762950 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:06.762951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:06.762951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:06.763214 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:06.763402 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:06.763485 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 12:22:06.763736 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260925 12:22:06.784662 9463 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:22:06.785231 9463 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:06.798384 9463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Generated new TSK 0
I20260925 12:22:06.798630 9463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:22:06.830396 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:06.938859 9476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:06.939078 9476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:06.939101 9476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:06.940217 9476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:06.940253 9476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:06.940279 9476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:06.941650 9476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:06.941854 9476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:06.942031 9476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:06.946991 9484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:06.947088 9476 server_base.cc:1034] running on GCE node
W20260925 12:22:06.947095 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:06.947106 9482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:06.947424 9476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:06.947676 9476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:06.948839 9476 hybrid_clock.cc:648] HybridClock initialized: now 1790338926948824 us; error 43 us; skew 500 ppm
Sep 25 12:22:06 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:06.951361 9476 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:22:06.951941 9476 webserver.cc:460] Webserver started at http://127.1.47.129:42045/ using document root <none> and password file <none>
I20260925 12:22:06.952183 9476 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:06.952250 9476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:06.952404 9476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:06.953441 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "70defa9c8d484c0da13dd0f1bc19d5b4"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "df6bb31065625a5d7eeb1b8bcb0fbf29"
server_key_iv: "3f960dfcd1585da0d14efb38e30a9e50"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.953807 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "70defa9c8d484c0da13dd0f1bc19d5b4"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "df6bb31065625a5d7eeb1b8bcb0fbf29"
server_key_iv: "3f960dfcd1585da0d14efb38e30a9e50"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.955132 9476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:22:06.955914 9491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:06.956106 9476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:06.956210 9476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "70defa9c8d484c0da13dd0f1bc19d5b4"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "df6bb31065625a5d7eeb1b8bcb0fbf29"
server_key_iv: "3f960dfcd1585da0d14efb38e30a9e50"
server_key_version: "encryptionkey@0"
I20260925 12:22:06.956280 9476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:06.971101 9476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:06.973284 9476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:06.973402 9476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:06.973686 9476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:06.974040 9476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:06.974105 9476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:06.974148 9476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:06.974175 9476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:06.980237 9476 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:40499
I20260925 12:22:06.980285 9604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:40499 every 8 connection(s)
I20260925 12:22:06.980640 9476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:22:06.986405 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9476
I20260925 12:22:06.986554 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:22:06.986770 1214 external_mini_cluster.cc:1428] Setting key f541993a4f48707754c131a1e1259503
Sep 25 12:22:06 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:06.991349 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:06.981399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:43043 (local address 127.1.47.190:46377)
0925 12:22:06.981466 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:06.981469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:06.981682 (+ 213us) server_negotiation.cc:407] Connection header received
0925 12:22:06.982233 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:06.982235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:06.982250 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:06.982310 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:06.983276 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.983847 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.984924 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.985150 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.985236 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:06.987922 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:06.987923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:06.987924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:06.987933 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:06.989494 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:06.990075 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:06.990077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:06.990078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:06.990139 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:06.990468 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:06.990469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:06.990469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:06.990621 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:06.990691 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:06.991091 (+ 400us) server_negotiation.cc:299] Negotiation successful
0925 12:22:06.991196 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:22:06.991544 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:06.981385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:43043)
0925 12:22:06.981550 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:06.981588 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:06.982123 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:06.982392 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:06.982396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:06.982972 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:06.983171 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:06.983175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.983984 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.983986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:06.984808 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:06.984810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:06.985247 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:06.985248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:06.985380 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:06.985939 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:06.985947 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:06.987806 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:06.989582 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:06.989589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:06.989592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:06.989981 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:06.990233 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:06.990234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:06.990234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:06.990390 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:06.990729 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:06.990735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:06.990919 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 12:22:06.991042 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 12:22:06.991294 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260925 12:22:06.992125 9605 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46377
I20260925 12:22:06.992246 9605 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:06.992472 9605 heartbeater.cc:511] Master 127.1.47.190:46377 requested a full tablet report, sending...
I20260925 12:22:06.993099 9414 ts_manager.cc:194] Registered new tserver with Master: 70defa9c8d484c0da13dd0f1bc19d5b4 (127.1.47.129:40499)
I20260925 12:22:06.993822 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:43043
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:22:07.015677 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:07.099121 9610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.099421 9610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.099462 9610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.100785 9610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.100880 9610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.100929 9610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:07.102525 9610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.103080 9610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.103400 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.108927 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.108929 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.108927 9615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.109382 9610 server_base.cc:1034] running on GCE node
I20260925 12:22:07.109632 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.109900 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.111100 9610 hybrid_clock.cc:648] HybridClock initialized: now 1790338927111078 us; error 40 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.114226 9610 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:22:07.114744 9610 webserver.cc:460] Webserver started at http://127.1.47.130:38595/ using document root <none> and password file <none>
I20260925 12:22:07.115010 9610 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.115085 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.115257 9610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:07.116528 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "92af60144d634b1db7c6f705f314ce0f"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "10e220a3f001bac707f7e914798af900"
server_key_iv: "729256a68f955ecf2bfce9367cdceb76"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.116951 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "92af60144d634b1db7c6f705f314ce0f"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "10e220a3f001bac707f7e914798af900"
server_key_iv: "729256a68f955ecf2bfce9367cdceb76"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.118432 9610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 12:22:07.119376 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.119654 9610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:07.119798 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "92af60144d634b1db7c6f705f314ce0f"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "10e220a3f001bac707f7e914798af900"
server_key_iv: "729256a68f955ecf2bfce9367cdceb76"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.119930 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.133553 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.136394 9610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.136570 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.136873 9610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:07.137246 9610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:07.137310 9610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.137359 9610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:07.137413 9610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.144114 9610 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:44583
I20260925 12:22:07.144167 9738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:44583 every 8 connection(s)
I20260925 12:22:07.144541 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:22:07.151139 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9610
I20260925 12:22:07.151273 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:07.151489 1214 external_mini_cluster.cc:1428] Setting key 3ac80a89da2b90ed2dddc33e53a0d32a
I20260925 12:22:07.155367 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.145331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:34479 (local address 127.1.47.190:46377)
0925 12:22:07.145431 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.145433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.145590 (+ 157us) server_negotiation.cc:407] Connection header received
0925 12:22:07.146149 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.146150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.146161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.146234 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.147193 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.147740 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.148835 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.149066 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.149141 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.152061 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:07.152062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:07.152063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:07.152071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:07.153773 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.154240 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.154242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.154242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.154305 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.154639 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.154640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.154640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.154792 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:07.154851 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:07.155164 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 12:22:07.155261 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 12:22:07.155617 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.145251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:34479)
0925 12:22:07.145435 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.145498 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.146049 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.146291 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.146294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.146880 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.147085 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.147089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.147826 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.147826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.148725 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.148727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.149179 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.149180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.149308 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.150017 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.150028 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.151956 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:07.153877 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:07.153886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:07.153891 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:07.154154 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:07.154391 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:07.154393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:07.154393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:07.154551 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:07.154861 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:07.154869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:07.155026 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 12:22:07.155145 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 12:22:07.155323 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260925 12:22:07.156138 9739 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46377
I20260925 12:22:07.156262 9739 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:07.156495 9739 heartbeater.cc:511] Master 127.1.47.190:46377 requested a full tablet report, sending...
I20260925 12:22:07.156900 9414 ts_manager.cc:194] Registered new tserver with Master: 92af60144d634b1db7c6f705f314ce0f (127.1.47.130:44583)
I20260925 12:22:07.157277 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:34479
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:22:07.181178 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:07.264070 9744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.264309 9744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.264348 9744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.265529 9744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.265599 9744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.265643 9744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:07.267115 9744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.267400 9744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.267634 9744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.273345 9750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.273386 9749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.273456 9744 server_base.cc:1034] running on GCE node
W20260925 12:22:07.273350 9752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.273800 9744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.274056 9744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.275205 9744 hybrid_clock.cc:648] HybridClock initialized: now 1790338927275179 us; error 56 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.277802 9744 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:22:07.278308 9744 webserver.cc:460] Webserver started at http://127.1.47.131:41295/ using document root <none> and password file <none>
I20260925 12:22:07.278556 9744 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.278627 9744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.278784 9744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:07.279842 9744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "b419ec37367f483bafebce4b39a7add2"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "a24019cc7c6b350233c908f0870c11b6"
server_key_iv: "260a46c28dbe41936882e84a27ec855d"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.280275 9744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "b419ec37367f483bafebce4b39a7add2"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "a24019cc7c6b350233c908f0870c11b6"
server_key_iv: "260a46c28dbe41936882e84a27ec855d"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.281622 9744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:07.282430 9759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.282624 9744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:07.282720 9744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "b419ec37367f483bafebce4b39a7add2"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "a24019cc7c6b350233c908f0870c11b6"
server_key_iv: "260a46c28dbe41936882e84a27ec855d"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.282812 9744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.300951 9744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.303119 9744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.303296 9744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.303689 9744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:07.304143 9744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:07.304203 9744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.304239 9744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:07.304268 9744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.310531 9744 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:44397
I20260925 12:22:07.310586 9872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:44397 every 8 connection(s)
I20260925 12:22:07.310933 9744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:22:07.316386 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9744
I20260925 12:22:07.316465 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:22:07.316641 1214 external_mini_cluster.cc:1428] Setting key 886a33e656411f2819e322daad263b9c
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:07.320236 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.311904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:37659 (local address 127.1.47.190:46377)
0925 12:22:07.311965 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.311967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.312128 (+ 161us) server_negotiation.cc:407] Connection header received
0925 12:22:07.312629 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.312630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.312641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.312709 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.313616 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.314111 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.315089 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.315282 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.315343 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.317735 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:07.317736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:07.317736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:07.317742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:07.318946 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.319307 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.319308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.319309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.319356 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.319576 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.319576 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.319577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.319674 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:07.319724 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:07.320083 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 12:22:07.320175 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:22:07.320438 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.311795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:37659)
0925 12:22:07.311998 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.312041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.312521 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.312767 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.312770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.313325 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.313531 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.313536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.314190 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.314190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.314993 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.314995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.315359 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.315360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.315473 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.316001 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.316009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.317644 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:07.319017 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:07.319023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:07.319025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:07.319239 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:07.319419 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:07.319420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:07.319420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:07.319522 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:07.319757 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:07.319763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:07.319962 (+ 199us) client_negotiation.cc:769] Sending connection context
0925 12:22:07.320068 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 12:22:07.320190 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260925 12:22:07.320885 9873 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46377
I20260925 12:22:07.320988 9873 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:07.321161 9873 heartbeater.cc:511] Master 127.1.47.190:46377 requested a full tablet report, sending...
I20260925 12:22:07.321538 9414 ts_manager.cc:194] Registered new tserver with Master: b419ec37367f483bafebce4b39a7add2 (127.1.47.131:44397)
I20260925 12:22:07.321870 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:37659
I20260925 12:22:07.328631 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
Entry for principal oryx/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab.
WARNING: no policy specified for oryx/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
Entry for principal oryx/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab.
WARNING: no policy specified for oryx/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
Entry for principal oryx/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab.
I20260925 12:22:07.399695 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9476
I20260925 12:22:07.404412 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9610
I20260925 12:22:07.408600 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9744
I20260925 12:22:07.412846 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9382
I20260925 12:22:07.417858 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46377
--webserver_interface=127.1.47.190
--webserver_port=38727
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:07.497925 9882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.498135 9882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.498158 9882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.499334 9882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:07.499411 9882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.499428 9882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.499439 9882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:07.499452 9882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:07.501026 9882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=38727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.501284 9882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.501546 9882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.506300 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.506258 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.506254 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.506557 9882 server_base.cc:1034] running on GCE node
I20260925 12:22:07.506804 9882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.507089 9882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.508229 9882 hybrid_clock.cc:648] HybridClock initialized: now 1790338927508198 us; error 44 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.518435 9882 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:22:07.518999 9882 webserver.cc:460] Webserver started at http://127.1.47.190:38727/ using document root <none> and password file <none>
I20260925 12:22:07.519306 9882 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.519404 9882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.520969 9882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:07.521801 9897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.522006 9882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:07.522119 9882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "7d77cd4f53f84ad6b07e5e60b036673c"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "b11f76ed3cc36eefa0d8f93eb8df3fae"
server_key_iv: "36baebb01b69eaec7c3bef3d8a0c9ebc"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.522488 9882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.535626 9882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.537926 9882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.538113 9882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.542299 9882 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46377
I20260925 12:22:07.542330 9949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46377 every 8 connection(s)
I20260925 12:22:07.542737 9882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:07.544255 9950 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:22:07.544899 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Bootstrap starting.
I20260925 12:22:07.552752 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9882
I20260925 12:22:07.552817 9950 log.cc:824] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:07.553061 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:40499
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:46377
--webserver_port=42045
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:22:07.554929 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:22:07.555217 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Bootstrap complete.
I20260925 12:22:07.557623 9950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:07.557873 9950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d77cd4f53f84ad6b07e5e60b036673c, State: Initialized, Role: FOLLOWER
I20260925 12:22:07.558089 9950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:07.558185 9950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:07.558238 9950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:07.558286 9950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:22:07.559290 9950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:07.559412 9950 leader_election.cc:302] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d77cd4f53f84ad6b07e5e60b036673c; no voters:
I20260925 12:22:07.559620 9950 leader_election.cc:288] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 12:22:07.559659 9954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 2 FOLLOWER]: Leader election won for term 2
I20260925 12:22:07.559772 9954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [term 2 LEADER]: Becoming Leader. State: Replica: 7d77cd4f53f84ad6b07e5e60b036673c, State: Running, Role: LEADER
I20260925 12:22:07.559885 9954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } }
I20260925 12:22:07.559923 9950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:07.560166 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d77cd4f53f84ad6b07e5e60b036673c. Latest consensus state: current_term: 2 leader_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } } }
I20260925 12:22:07.560241 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:07.560173 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77cd4f53f84ad6b07e5e60b036673c" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46377 } } }
I20260925 12:22:07.560298 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:07.560442 9958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:07.561230 9958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:07.561368 9958 catalog_manager.cc:1259] Loaded cluster ID: ae7b799ebc3c49c7899f7a32de27c3f9
I20260925 12:22:07.561406 9958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:22:07.562667 9958 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:07.562922 9958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d77cd4f53f84ad6b07e5e60b036673c: Loaded TSK: 0
I20260925 12:22:07.563140 9958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 12:22:07.636590 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.636852 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.636893 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.638118 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.638195 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.638253 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:07.639765 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.129.keytab
--principal=oryx/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:40499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=42045
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.640075 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.640331 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.646644 9979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.646638 9977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.646634 9976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.647079 9953 server_base.cc:1034] running on GCE node
I20260925 12:22:07.647341 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.647688 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.648909 9953 hybrid_clock.cc:648] HybridClock initialized: now 1790338927648894 us; error 48 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.652134 9953 init.cc:377] Logged in from keytab as oryx/127.1.47.129@KRBTEST.COM (short username oryx)
I20260925 12:22:07.652593 9953 webserver.cc:460] Webserver started at http://127.1.47.129:42045/ using document root <none> and password file <none>
I20260925 12:22:07.652850 9953 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.652936 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.654479 9953 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:22:07.655215 9986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.655436 9953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:07.655547 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "70defa9c8d484c0da13dd0f1bc19d5b4"
format_stamp: "Formatted at 2026-09-25 12:22:06 on dist-test-slave-l9ww"
server_key: "df6bb31065625a5d7eeb1b8bcb0fbf29"
server_key_iv: "3f960dfcd1585da0d14efb38e30a9e50"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.655980 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.667687 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.670209 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.670333 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.670619 9953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:07.671013 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:07.671068 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.671108 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:07.671133 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.678359 9953 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:40499
I20260925 12:22:07.678387 10099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:40499 every 8 connection(s)
I20260925 12:22:07.678812 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.688653 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 9953
I20260925 12:22:07.688998 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:44583
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:46377
--webserver_port=38595
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:22:07.689152 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.679755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:47931)
0925 12:22:07.679970 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.680019 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.680518 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.681058 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.681063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.681869 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.682156 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.682162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.683467 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.683468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.684874 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.684878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.685548 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.685549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.685696 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.686320 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.686330 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.688914 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":127,"threads_started":1}
W20260925 12:22:07.689453 10100 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:46377 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.47.190:46377: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
W20260925 12:22:07.690044 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.679809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:47931 (local address 127.1.47.190:46377)
0925 12:22:07.680679 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.680686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.680713 (+ 27us) server_negotiation.cc:407] Connection header received
0925 12:22:07.680811 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.680815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.680853 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.680948 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.682336 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.683354 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.684981 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.685447 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.685548 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.689831 (+ 4283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:47931: BlockingRecv error: recv got EOF from 127.1.47.129:47931 (error 108)
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":129,"threads_started":1}
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.694746 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.689686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:36589)
0925 12:22:07.689907 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.689918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.689960 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.690485 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.690487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.690626 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.690674 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.690675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.691329 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.691330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.691941 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.691943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.692306 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.692307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.692369 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.692853 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.692858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.694659 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204}
W20260925 12:22:07.694808 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.689770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:36589 (local address 127.1.47.190:46377)
0925 12:22:07.690223 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.690225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.690233 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:07.690387 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.690388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.690400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.690467 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.690754 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.691242 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.692019 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.692230 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.692309 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.694738 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:36589: BlockingRecv error: recv got EOF from 127.1.47.129:36589 (error 108)
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":103,"threads_started":1}
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.699438 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.695253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:41589)
0925 12:22:07.695326 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.695337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.695384 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.695515 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.695516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.695643 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.695672 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.695673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.696376 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.696377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.696874 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.696875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.697248 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.697250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.697310 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.697749 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.697754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.699353 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 12:22:07.699484 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.695298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41589 (local address 127.1.47.190:46377)
0925 12:22:07.695334 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.695337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.695369 (+ 32us) server_negotiation.cc:407] Connection header received
0925 12:22:07.695442 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.695443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.695453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.695506 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.695753 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.696299 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.696968 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.697166 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.697249 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.699420 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:41589: BlockingRecv error: recv got EOF from 127.1.47.129:41589 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 12:22:07.699810 10100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:22:07.774554 10105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.774773 10105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.774801 10105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.775974 10105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.776010 10105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.776036 10105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:07.777385 10105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.130.keytab
--principal=oryx/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:44583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=38595
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.777565 10105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.777738 10105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.782685 10113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.782686 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.782725 10105 server_base.cc:1034] running on GCE node
W20260925 12:22:07.782716 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.783082 10105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.783340 10105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.784507 10105 hybrid_clock.cc:648] HybridClock initialized: now 1790338927784512 us; error 45 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.786926 10105 init.cc:377] Logged in from keytab as oryx/127.1.47.130@KRBTEST.COM (short username oryx)
I20260925 12:22:07.787343 10105 webserver.cc:460] Webserver started at http://127.1.47.130:38595/ using document root <none> and password file <none>
I20260925 12:22:07.787559 10105 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.787636 10105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.789136 10105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:07.789687 10120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.789857 10105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:07.789953 10105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "92af60144d634b1db7c6f705f314ce0f"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "10e220a3f001bac707f7e914798af900"
server_key_iv: "729256a68f955ecf2bfce9367cdceb76"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.790330 10105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.799574 10105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.801386 10105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.801545 10105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.801774 10105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:07.802065 10105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:07.802119 10105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.802158 10105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:07.802186 10105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.808265 10105 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:44583
I20260925 12:22:07.808339 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:44583 every 8 connection(s)
I20260925 12:22:07.808694 10105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.815245 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10105
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.815605 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:44397
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:46377
--webserver_port=41295
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:22:07.816547 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.809544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:52621)
0925 12:22:07.809690 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.809726 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.810183 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.810411 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.810415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.810970 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.811164 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.811168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.811950 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.811951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.812834 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.812837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.813181 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.813182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.813502 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.814018 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.814027 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.816398 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260925 12:22:07.816587 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.809544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:52621 (local address 127.1.47.190:46377)
0925 12:22:07.809588 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.809590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.809795 (+ 205us) server_negotiation.cc:407] Connection header received
0925 12:22:07.810283 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.810285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.810300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.810357 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.811258 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.811755 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.812918 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.813102 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.813162 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.816499 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:52621: BlockingRecv error: recv got EOF from 127.1.47.130:52621 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 12:22:07.816855 10234 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:46377 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.47.190:46377: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.821588 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.817088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:37351)
0925 12:22:07.817140 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.817151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.817189 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.817312 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.817313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.817466 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.817522 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.817524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.818115 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.818115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.818639 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.818640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.819123 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.819123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.819168 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.819557 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.819561 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.821490 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:22:07.821616 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.817147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:37351 (local address 127.1.47.190:46377)
0925 12:22:07.817188 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.817190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.817207 (+ 17us) server_negotiation.cc:407] Connection header received
0925 12:22:07.817235 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.817236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.817245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.817295 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.817599 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.818049 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.818731 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.819048 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.819122 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.821571 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:37351: BlockingRecv error: recv got EOF from 127.1.47.130:37351 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.826220 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.821929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:58053)
0925 12:22:07.822053 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.822064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.822130 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.822271 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.822271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.822402 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.822426 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.822428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.823035 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.823036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.823536 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.823538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.823886 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.823887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.823934 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.824354 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.824359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.826129 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110}
W20260925 12:22:07.826251 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.821992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:58053 (local address 127.1.47.190:46377)
0925 12:22:07.822051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.822053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.822116 (+ 63us) server_negotiation.cc:407] Connection header received
0925 12:22:07.822188 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.822189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.822198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.822268 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.822500 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.822957 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.823616 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.823802 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.823915 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.826194 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:58053: BlockingRecv error: recv got EOF from 127.1.47.130:58053 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 12:22:07.826347 10234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:22:07.899647 10237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:07.899951 10237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:07.900025 10237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:07.901322 10237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:07.901396 10237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:07.901444 10237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:07.902937 10237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/krb5kdc/oryx_127.1.47.131.keytab
--principal=oryx/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:44397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=41295
--tserver_master_addrs=127.1.47.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:07.903205 10237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:07.903440 10237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:07.908435 10242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:07.908427 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.908520 10237 server_base.cc:1034] running on GCE node
W20260925 12:22:07.908427 10245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:07.908799 10237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:07.909041 10237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:07.910205 10237 hybrid_clock.cc:648] HybridClock initialized: now 1790338927910194 us; error 29 us; skew 500 ppm
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338927, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:07.912738 10237 init.cc:377] Logged in from keytab as oryx/127.1.47.131@KRBTEST.COM (short username oryx)
I20260925 12:22:07.913234 10237 webserver.cc:460] Webserver started at http://127.1.47.131:41295/ using document root <none> and password file <none>
I20260925 12:22:07.913465 10237 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:07.913544 10237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:07.915129 10237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:22:07.915896 10252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.916126 10237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:07.916213 10237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "b419ec37367f483bafebce4b39a7add2"
format_stamp: "Formatted at 2026-09-25 12:22:07 on dist-test-slave-l9ww"
server_key: "a24019cc7c6b350233c908f0870c11b6"
server_key_iv: "260a46c28dbe41936882e84a27ec855d"
server_key_version: "encryptionkey@0"
I20260925 12:22:07.916582 10237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:07.927556 10237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:07.929608 10237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:07.929765 10237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:07.929996 10237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:07.930311 10237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:07.930365 10237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.930418 10237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:07.930444 10237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:07.936265 10237 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:44397
I20260925 12:22:07.936326 10365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:44397 every 8 connection(s)
I20260925 12:22:07.936705 10237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:22:07.941246 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10237
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 12:22:07.945254 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.937519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:40441 (local address 127.1.47.190:46377)
0925 12:22:07.937592 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.937595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.937849 (+ 254us) server_negotiation.cc:407] Connection header received
0925 12:22:07.938370 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.938385 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.938395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.938447 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.939460 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.940012 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.941157 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.941404 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.941479 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.945191 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:40441: BlockingRecv error: recv got EOF from 127.1.47.131:40441 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:22:07.945346 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.937503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:40441)
0925 12:22:07.937701 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.937752 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.938279 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.938530 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.938533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.939172 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.939379 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.939384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.940108 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.940109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.941054 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.941057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.941488 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.941489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.941657 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.942471 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.942480 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.945129 (+ 2649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
W20260925 12:22:07.945536 10366 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:46377 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.47.190:46377: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.949654 10104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.941778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.1.47.190:46377)
0925 12:22:07.941841 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.941844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.941856 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:22:07.941946 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.941947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.941966 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.942048 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.942678 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.943346 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.944452 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.944648 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.944701 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.946222 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:07.946227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:07.946228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:07.946243 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:07.947957 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.948387 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.948390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.948391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.948492 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:07.948703 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:07.948704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:07.948704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:07.949026 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:07.949177 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:07.949374 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 12:22:07.949588 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.950270 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.945796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:46779)
0925 12:22:07.945842 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.945852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.945891 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.946014 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.946014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.946146 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.946188 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.946190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.946994 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.946995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.947508 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.947510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.947900 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.947901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.947964 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.948339 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.948344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.950197 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 12:22:07.950390 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.945861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:46779 (local address 127.1.47.190:46377)
0925 12:22:07.945922 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.945924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.945932 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:07.945953 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.945954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.945962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.946012 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.946264 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.946895 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.947592 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.947804 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.947920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.950330 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:46779: BlockingRecv error: recv got EOF from 127.1.47.131:46779 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:07 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:07.954598 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:07.950535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:48035)
0925 12:22:07.950611 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:07.950621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:07.950662 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:07.950777 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:07.950777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:07.950883 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:07.950910 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.950911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.951488 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.951489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.952150 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:07.952152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.952469 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.952470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:07.952510 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.952979 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:07.952985 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:07.954551 (+ 1566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 12:22:07.954717 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:07.950589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:48035 (local address 127.1.47.190:46377)
0925 12:22:07.950620 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:07.950621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:07.950650 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:22:07.950707 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:07.950707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:07.950718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:07.950772 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:07.950965 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.951400 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.952226 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:07.952400 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:07.952454 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:07.954625 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:48035: BlockingRecv error: recv got EOF from 127.1.47.131:48035 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 12:22:07.954728 10366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:08.705507 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:08.700289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:58359)
0925 12:22:08.700435 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:08.700452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:08.700530 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:08.700742 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:08.700743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:08.700910 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:08.700958 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.700959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.701696 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.701697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.702253 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.702255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.702687 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.702689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.702743 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.703478 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:08.703482 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:08.705442 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
W20260925 12:22:08.705654 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:08.700391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:58359 (local address 127.1.47.190:46377)
0925 12:22:08.700586 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:08.700588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:08.700599 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:22:08.700626 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:08.700627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:08.700637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:08.700718 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:08.701064 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.701576 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.702357 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.702556 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.702643 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.705560 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:58359: BlockingRecv error: recv got EOF from 127.1.47.129:58359 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:08.831717 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:08.826724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:46015)
0925 12:22:08.826924 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:08.826937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:08.827008 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:08.827135 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:08.827136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:08.827353 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:08.827401 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.827404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.828202 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.828204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.828784 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.828786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.829185 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.829186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.829247 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.829761 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:08.829766 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:08.831612 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":110,"threads_started":1}
W20260925 12:22:08.831773 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:08.826827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:46015 (local address 127.1.47.190:46377)
0925 12:22:08.826913 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:08.826915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:08.826989 (+ 74us) server_negotiation.cc:407] Connection header received
0925 12:22:08.827062 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:08.827063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:08.827076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:08.827121 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:08.827507 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.828107 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.828897 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.829095 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.829183 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.831706 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:46015: BlockingRecv error: recv got EOF from 127.1.47.130:46015 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:08 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:08.959990 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:08.955226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:42185)
0925 12:22:08.955370 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:08.955382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:08.955451 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:08.955604 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:08.955606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:08.955780 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:08.955826 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.955828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.956567 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.956568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.957123 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:08.957125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.957493 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.957494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:08.957530 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.957994 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:08.957999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:08.959854 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260925 12:22:08.960021 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:08.955309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:42185 (local address 127.1.47.190:46377)
0925 12:22:08.955377 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:08.955379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:08.955448 (+ 69us) server_negotiation.cc:407] Connection header received
0925 12:22:08.955504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:08.955505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:08.955518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:08.955587 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:08.955972 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.956497 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.957220 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:08.957417 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:08.957493 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:08.959956 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:42185: BlockingRecv error: recv got EOF from 127.1.47.131:42185 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:09.710721 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:09.705966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:56761)
0925 12:22:09.706136 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:09.706146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:09.706202 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:09.706334 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:09.706335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:09.706500 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:09.706551 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.706552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.707249 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.707249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.707772 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.707773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.708169 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.708169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.708211 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.708799 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:09.708803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:09.710609 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
W20260925 12:22:09.710763 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:09.706076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:56761 (local address 127.1.47.190:46377)
0925 12:22:09.706179 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:09.706181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:09.706190 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:09.706247 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:09.706247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:09.706258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:09.706316 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:09.706617 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.707179 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.707843 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.708105 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.708160 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.710688 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:56761: BlockingRecv error: recv got EOF from 127.1.47.129:56761 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:09.837543 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:09.832348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:33983)
0925 12:22:09.832515 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:09.832536 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:09.832603 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:09.832800 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:09.832801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:09.832991 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:09.833042 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.833044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.833722 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.833724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.834545 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.834548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.834936 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.834937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.834985 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.835464 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:09.835468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:09.837485 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
W20260925 12:22:09.837769 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:09.832458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:33983 (local address 127.1.47.190:46377)
0925 12:22:09.832531 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:09.832533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:09.832603 (+ 70us) server_negotiation.cc:407] Connection header received
0925 12:22:09.832658 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:09.832659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:09.832669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:09.832738 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:09.833119 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.833626 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.834630 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.834867 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.834920 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.837672 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:33983: BlockingRecv error: recv got EOF from 127.1.47.130:33983 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:09 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:09.965138 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:09.960544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:39853)
0925 12:22:09.960717 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:09.960729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:09.960790 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:09.960961 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:09.960962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:09.961148 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:09.961203 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.961206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.961935 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.961936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.962454 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:09.962455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.962833 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.962834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:09.962872 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.963317 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:09.963321 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:09.965070 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
W20260925 12:22:09.965267 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:09.960633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:39853 (local address 127.1.47.190:46377)
0925 12:22:09.960701 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:09.960703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:09.960789 (+ 86us) server_negotiation.cc:407] Connection header received
0925 12:22:09.960854 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:09.960854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:09.960867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:09.960910 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:09.961281 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.961829 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.962556 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:09.962764 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:09.962828 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:09.965164 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:39853: BlockingRecv error: recv got EOF from 127.1.47.131:39853 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:10.716758 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:10.711337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:59195)
0925 12:22:10.711529 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:10.711541 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:10.711604 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:10.711793 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:10.711793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:10.711978 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:10.712026 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.712028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.712814 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.712815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.713429 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.713431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.713996 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.713997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.714067 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.714610 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:10.714623 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:10.716696 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":129,"threads_started":1}
W20260925 12:22:10.716943 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:10.711445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:59195 (local address 127.1.47.190:46377)
0925 12:22:10.711583 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:10.711586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:10.711598 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:22:10.711695 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:10.711696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:10.711708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:10.711782 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:10.712130 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.712708 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.713561 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.713882 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.713969 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.716855 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:59195: BlockingRecv error: recv got EOF from 127.1.47.129:59195 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:10.843395 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:10.838195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:32969)
0925 12:22:10.838384 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:10.838404 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:10.838492 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:10.838663 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:10.838664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:10.838846 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:10.838899 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.838901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.839600 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.839601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.840161 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.840163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.840575 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.840575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.840637 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.841197 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:10.841204 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:10.843320 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
W20260925 12:22:10.843508 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:10.838253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:32969 (local address 127.1.47.190:46377)
0925 12:22:10.838358 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:10.838360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:10.838474 (+ 114us) server_negotiation.cc:407] Connection header received
0925 12:22:10.838569 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:10.838571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:10.838584 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:10.838659 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:10.838988 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.839509 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.840251 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.840505 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.840576 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.843435 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:32969: BlockingRecv error: recv got EOF from 127.1.47.130:32969 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:10 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:10.970750 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:10.965663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:33359)
0925 12:22:10.965827 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:10.965838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:10.965909 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:10.966224 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:10.966226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:10.966435 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:10.966512 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.966515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.967276 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.967277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.967824 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:10.967826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.968322 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.968323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:10.968383 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.968880 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:10.968885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:10.970689 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
W20260925 12:22:10.970916 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:10.965757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:33359 (local address 127.1.47.190:46377)
0925 12:22:10.965816 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:10.965820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:10.965915 (+ 95us) server_negotiation.cc:407] Connection header received
0925 12:22:10.965987 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:10.965988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:10.966001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:10.966105 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:10.966608 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.967197 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.967968 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:10.968233 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:10.968313 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:10.970846 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:33359: BlockingRecv error: recv got EOF from 127.1.47.131:33359 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:11.722506 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:11.717382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:54511)
0925 12:22:11.717536 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:11.717550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:11.717619 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:11.717849 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:11.717850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:11.718111 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:11.718176 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.718179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.718938 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.718939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.719566 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.719568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.719974 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.719975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.720055 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.720554 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:11.720559 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:11.722442 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
W20260925 12:22:11.722633 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:11.717456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:54511 (local address 127.1.47.190:46377)
0925 12:22:11.717592 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:11.717594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:11.717605 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:22:11.717682 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:11.717683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:11.717694 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:11.717779 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:11.718256 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.718845 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.719643 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.719845 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.719916 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.722547 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:54511: BlockingRecv error: recv got EOF from 127.1.47.129:54511 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:11.849011 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:11.844082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:42505)
0925 12:22:11.844281 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:11.844293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:11.844393 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:11.844559 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:11.844560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:11.844745 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:11.844790 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.844792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.845482 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.845483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.846017 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.846018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.846471 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.846472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.846528 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.847037 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:11.847042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:11.848913 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
W20260925 12:22:11.849061 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:11.844174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:42505 (local address 127.1.47.190:46377)
0925 12:22:11.844255 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:11.844257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:11.844375 (+ 118us) server_negotiation.cc:407] Connection header received
0925 12:22:11.844462 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:11.844463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:11.844476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:11.844553 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:11.844888 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.845395 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.846102 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.846395 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.846474 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.848995 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:42505: BlockingRecv error: recv got EOF from 127.1.47.130:42505 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:11 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:11.976485 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:11.971435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:56651)
0925 12:22:11.971551 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:11.971629 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:11.971719 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:11.971924 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:11.971925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:11.972133 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:11.972180 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.972182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.972901 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.972902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.973565 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:11.973568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.973964 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.973965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:11.974025 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.974521 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:11.974525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:11.976424 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
W20260925 12:22:11.976656 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:11.971527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:56651 (local address 127.1.47.190:46377)
0925 12:22:11.971578 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:11.971580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:11.971703 (+ 123us) server_negotiation.cc:407] Connection header received
0925 12:22:11.971789 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:11.971790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:11.971804 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:11.971886 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:11.972267 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.972775 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.973662 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:11.973878 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:11.973953 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:11.976571 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:56651: BlockingRecv error: recv got EOF from 127.1.47.131:56651 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:12.735107 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:12.723072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:50487)
0925 12:22:12.723184 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:12.723195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:12.723251 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:12.723472 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:12.723473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:12.723650 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:12.723699 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.723700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.724403 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.724404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.724993 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.724995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.725341 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.725342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.725400 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.732850 (+ 7450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:12.732856 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:12.735044 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
W20260925 12:22:12.735263 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:12.723141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:50487 (local address 127.1.47.190:46377)
0925 12:22:12.723263 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:12.723265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:12.723275 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:12.723361 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:12.723362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:12.723376 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:12.723458 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:12.723770 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.724330 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.725064 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.725252 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.725304 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.735174 (+ 9870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:50487: BlockingRecv error: recv got EOF from 127.1.47.129:50487 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:12.854605 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:12.849610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:35893)
0925 12:22:12.849765 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:12.849780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:12.849854 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:12.849983 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:12.849984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:12.850157 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:12.850203 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.850205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.850891 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.850893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.851451 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.851453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.851848 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.851849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.851937 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.852382 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:12.852387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:12.854545 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
W20260925 12:22:12.854808 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:12.849744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:35893 (local address 127.1.47.190:46377)
0925 12:22:12.849792 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:12.849795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:12.849847 (+ 52us) server_negotiation.cc:407] Connection header received
0925 12:22:12.849916 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:12.849917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:12.849930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:12.849980 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:12.850291 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.850783 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.851555 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.851754 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.851847 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.854741 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:35893: BlockingRecv error: recv got EOF from 127.1.47.130:35893 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:12 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:12.981549 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:12.976952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:44273)
0925 12:22:12.977104 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:12.977123 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:12.977180 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:12.977326 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:12.977327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:12.977489 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:12.977539 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.977540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.978246 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.978247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.978844 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:12.978846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.979216 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.979218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:12.979261 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.979717 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:12.979722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:12.981489 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
W20260925 12:22:12.981693 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:12.977052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:44273 (local address 127.1.47.190:46377)
0925 12:22:12.977121 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:12.977123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:12.977180 (+ 57us) server_negotiation.cc:407] Connection header received
0925 12:22:12.977227 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:12.977228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:12.977238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:12.977300 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:12.977605 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.978112 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.978937 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:12.979123 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:12.979181 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:12.981611 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:44273: BlockingRecv error: recv got EOF from 127.1.47.131:44273 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:13.740398 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:13.735590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:43301)
0925 12:22:13.735762 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:13.735773 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:13.735824 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:13.736011 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:13.736011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:13.736174 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:13.736221 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.736222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.736904 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.736905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.737453 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.737454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.737790 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.737790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.737829 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.738379 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:13.738384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:13.740298 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260925 12:22:13.740458 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:13.735699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:43301 (local address 127.1.47.190:46377)
0925 12:22:13.735851 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:13.735878 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:13.735886 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:13.735926 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:13.735927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:13.735940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:13.736006 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:13.736315 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.736821 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.737536 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.737725 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.737780 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.740387 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:43301: BlockingRecv error: recv got EOF from 127.1.47.129:43301 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:13.859941 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:13.855154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:37937)
0925 12:22:13.855294 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:13.855305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:13.855383 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:13.855539 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:13.855540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:13.855733 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:13.855779 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.855781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.856518 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.856519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.857046 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.857048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.857439 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.857441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.857504 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.857990 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:13.857994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:13.859813 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
W20260925 12:22:13.859990 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:13.855323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:37937 (local address 127.1.47.190:46377)
0925 12:22:13.855397 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:13.855399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:13.855409 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:13.855441 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:13.855442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:13.855455 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:13.855531 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:13.855901 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.856426 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.857133 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.857353 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.857430 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.859926 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:37937: BlockingRecv error: recv got EOF from 127.1.47.130:37937 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:13 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:13.986661 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:13.982065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:51867)
0925 12:22:13.982208 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:13.982227 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:13.982287 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:13.982418 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:13.982419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:13.982591 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:13.982638 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.982640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.983265 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.983266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.983782 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:13.983784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.984156 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.984157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:13.984206 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.984665 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:13.984670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:13.986588 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
W20260925 12:22:13.986788 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:13.982149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:51867 (local address 127.1.47.190:46377)
0925 12:22:13.982212 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:13.982214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:13.982286 (+ 72us) server_negotiation.cc:407] Connection header received
0925 12:22:13.982332 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:13.982332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:13.982345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:13.982401 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:13.982707 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.983192 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.983853 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:13.984090 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:13.984145 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:13.986687 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:51867: BlockingRecv error: recv got EOF from 127.1.47.131:51867 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:14.745316 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:14.740938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:46653)
0925 12:22:14.741085 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:14.741096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:14.741144 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:14.741287 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:14.741287 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:14.741446 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:14.741494 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.741496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.742119 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.742119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.742636 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.742637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.742951 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.742952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.742991 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.743441 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:14.743445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:14.745218 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
W20260925 12:22:14.745378 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:14.741032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:46653 (local address 127.1.47.190:46377)
0925 12:22:14.741153 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:14.741155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:14.741164 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:14.741189 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:14.741190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:14.741206 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:14.741268 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:14.741562 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.742048 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.742705 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.742891 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.742940 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.745299 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:46653: BlockingRecv error: recv got EOF from 127.1.47.129:46653 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":84,"threads_started":1}
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:14.865121 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:14.860456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:47451)
0925 12:22:14.860578 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:14.860607 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:14.860664 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:14.860864 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:14.860865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:14.861043 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:14.861091 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.861093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.861780 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.861781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.862355 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.862357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.862763 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.862764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.862802 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.863309 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:14.863313 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:14.865062 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
W20260925 12:22:14.865253 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:14.860526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:47451 (local address 127.1.47.190:46377)
0925 12:22:14.860598 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:14.860600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:14.860651 (+ 51us) server_negotiation.cc:407] Connection header received
0925 12:22:14.860739 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:14.860740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:14.860751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:14.860853 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:14.861156 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.861708 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.862450 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.862687 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.862759 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.865156 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:47451: BlockingRecv error: recv got EOF from 127.1.47.130:47451 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:14 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:14.991972 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:14.987183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:47781)
0925 12:22:14.987349 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:14.987363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:14.987436 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:14.987592 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:14.987593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:14.987768 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:14.987827 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.987829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.988543 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.988545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.989083 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:14.989085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.989457 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.989458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:14.989526 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.989966 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:14.989971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:14.991914 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260925 12:22:14.992063 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:14.987309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:47781 (local address 127.1.47.190:46377)
0925 12:22:14.987371 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:14.987373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:14.987414 (+ 41us) server_negotiation.cc:407] Connection header received
0925 12:22:14.987497 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:14.987498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:14.987511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:14.987587 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:14.987953 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.988454 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.989185 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:14.989382 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:14.989459 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:14.991995 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:47781: BlockingRecv error: recv got EOF from 127.1.47.131:47781 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:15.750870 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:15.745854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:39125)
0925 12:22:15.746031 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:15.746043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:15.746095 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:15.746341 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:15.746342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:15.746528 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:15.746579 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.746580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.747353 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.747354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.747924 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.747926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.748295 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.748296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.748335 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:15.748806 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:15.748810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:15.750750 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
W20260925 12:22:15.750921 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:15.746009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:39125 (local address 127.1.47.190:46377)
0925 12:22:15.746168 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:15.746175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:15.746184 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:15.746208 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:15.746209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:15.746221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:15.746296 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:15.746671 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.747255 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.747999 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.748224 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.748288 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:15.750840 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:39125: BlockingRecv error: recv got EOF from 127.1.47.129:39125 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:15.870132 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:15.865596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:47785)
0925 12:22:15.865745 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:15.865757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:15.865838 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:15.865984 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:15.865984 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:15.866141 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:15.866190 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.866191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.866964 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.866965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.867491 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.867492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.867831 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.867831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.867924 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:15.868360 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:15.868365 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:15.870067 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":53,"threads_started":1}
W20260925 12:22:15.870208 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:15.865714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:47785 (local address 127.1.47.190:46377)
0925 12:22:15.865773 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:15.865780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:15.865820 (+ 40us) server_negotiation.cc:407] Connection header received
0925 12:22:15.865892 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:15.865893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:15.865907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:15.865954 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:15.866248 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.866891 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.867564 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.867753 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.867817 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:15.870159 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:47785: BlockingRecv error: recv got EOF from 127.1.47.130:47785 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:15 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:16.004298 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:15.992524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:57875)
0925 12:22:15.992676 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:15.992689 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:15.992763 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:15.992904 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:15.992905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:15.993068 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:15.993117 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.993119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.993844 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.993845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.994378 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:15.994380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.994765 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.994766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:15.994839 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.002292 (+ 7453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:16.002302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:16.004198 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
W20260925 12:22:16.004379 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:15.992613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:57875 (local address 127.1.47.190:46377)
0925 12:22:15.992680 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:15.992683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:15.992748 (+ 65us) server_negotiation.cc:407] Connection header received
0925 12:22:15.992811 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:15.992813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:15.992824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:15.992884 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:15.993180 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.993711 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.994469 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:15.994671 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:15.994764 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.004301 (+ 9537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:57875: BlockingRecv error: recv got EOF from 127.1.47.131:57875 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 12:22:16 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:16 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:16.756494 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:16.751377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:32899)
0925 12:22:16.751575 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:16.751589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:16.751663 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:16.751922 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:16.751923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:16.752194 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:16.752266 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:16.752269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.753033 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.753034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:16.753617 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:16.753619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.753950 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.753951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:16.753990 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.754439 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:16.754444 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:16.756435 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
W20260925 12:22:16.756659 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:16.751524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:32899 (local address 127.1.47.190:46377)
0925 12:22:16.751661 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:16.751663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:16.751673 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:16.751726 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:16.751727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:16.751738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:16.751818 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:16.752361 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.752944 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.753694 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.753887 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.753937 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.756543 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:32899: BlockingRecv error: recv got EOF from 127.1.47.129:32899 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
Sep 25 12:22:16 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:16 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:16.875758 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:16.870739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:53895)
0925 12:22:16.870927 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:16.870941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:16.871038 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:16.871221 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:16.871222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:16.871422 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:16.871472 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:16.871474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.872351 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.872352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:16.872890 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:16.872892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.873260 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.873261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:16.873325 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.873836 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:16.873841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:16.875649 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
W20260925 12:22:16.875806 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:16.870869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:53895 (local address 127.1.47.190:46377)
0925 12:22:16.870932 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:16.870935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:16.871017 (+ 82us) server_negotiation.cc:407] Connection header received
0925 12:22:16.871127 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:16.871128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:16.871148 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:16.871214 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:16.871564 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.872263 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.872979 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:16.873175 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:16.873260 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:16.875731 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:53895: BlockingRecv error: recv got EOF from 127.1.47.130:53895 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:17.009763 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:17.004802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:52869)
0925 12:22:17.005008 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:17.005020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:17.005122 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:17.005295 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:17.005296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:17.005482 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:17.005527 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.005529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.006222 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.006224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.006781 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.006783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.007161 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.007162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.007221 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.007753 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:17.007758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:17.009700 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
W20260925 12:22:17.009903 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:17.004886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:52869 (local address 127.1.47.190:46377)
0925 12:22:17.004992 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:17.004994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:17.005103 (+ 109us) server_negotiation.cc:407] Connection header received
0925 12:22:17.005198 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:17.005199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:17.005219 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:17.005287 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:17.005610 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.006129 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.006872 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.007064 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.007161 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.009818 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:52869: BlockingRecv error: recv got EOF from 127.1.47.131:52869 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:17.765830 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:17.757107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:56373)
0925 12:22:17.757230 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:17.757243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:17.757308 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:17.757505 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:17.757507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:17.757676 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:17.757734 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.757736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.758415 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.758416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.758964 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.758965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.759337 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.759338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.759399 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.763642 (+ 4243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:17.763647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:17.765765 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
W20260925 12:22:17.765962 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:17.757168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:56373 (local address 127.1.47.190:46377)
0925 12:22:17.757302 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:17.757304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:17.757312 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:22:17.757381 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:17.757382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:17.757394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:17.757497 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:17.757816 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.758330 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.759060 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.759265 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.759338 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.765857 (+ 6519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:56373: BlockingRecv error: recv got EOF from 127.1.47.129:56373 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:17 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:17.880970 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:17.876307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:55461)
0925 12:22:17.876533 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:17.876546 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:17.876613 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:17.876745 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:17.876745 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:17.876908 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:17.876955 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.876956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.877571 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.877572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.878084 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:17.878086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.878396 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.878397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:17.878446 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.878902 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:17.878907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:17.880870 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":174,"threads_started":1}
W20260925 12:22:17.881013 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:17.876372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:55461 (local address 127.1.47.190:46377)
0925 12:22:17.876422 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:17.876432 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:17.876614 (+ 182us) server_negotiation.cc:407] Connection header received
0925 12:22:17.876656 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:17.876657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:17.876670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:17.876726 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:17.877011 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.877503 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.878151 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:17.878336 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:17.878386 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:17.880950 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:55461: BlockingRecv error: recv got EOF from 127.1.47.130:55461 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:18.015021 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:18.010334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:54151)
0925 12:22:18.010473 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:18.010484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:18.010533 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:18.010655 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:18.010656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:18.010822 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:18.010870 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.010872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.011581 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.011581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.012265 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.012268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.012618 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.012619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.012664 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.013120 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:18.013125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:18.014921 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
W20260925 12:22:18.015070 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:18.010484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54151 (local address 127.1.47.190:46377)
0925 12:22:18.010537 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:18.010539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:18.010548 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:18.010570 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:18.010571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:18.010581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:18.010635 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:18.010969 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.011497 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.012342 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.012535 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.012602 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.015003 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:54151: BlockingRecv error: recv got EOF from 127.1.47.131:54151 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:18.771095 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:18.766386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:53837)
0925 12:22:18.766559 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:18.766572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:18.766638 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:18.766779 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:18.766780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:18.766964 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:18.767011 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.767013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.767646 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.767647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.768265 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.768267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.768681 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.768682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.768739 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.769248 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:18.769252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:18.771023 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
W20260925 12:22:18.771222 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:18.766445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:53837 (local address 127.1.47.190:46377)
0925 12:22:18.766614 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:18.766616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:18.766626 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:18.766697 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:18.766697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:18.766708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:18.766758 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:18.767075 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.767566 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.768363 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.768578 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.768655 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.771123 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:53837: BlockingRecv error: recv got EOF from 127.1.47.129:53837 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:18 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:18.887214 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:18.881574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:60029)
0925 12:22:18.881751 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:18.881764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:18.881838 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:18.882081 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:18.882082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:18.882367 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:18.882438 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.882441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.883453 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.883455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.884153 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:18.884156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.884530 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.884531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:18.884574 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.885133 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:18.885138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:18.887112 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
W20260925 12:22:18.887295 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:18.881705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:60029 (local address 127.1.47.190:46377)
0925 12:22:18.881763 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:18.881765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:18.881824 (+ 59us) server_negotiation.cc:407] Connection header received
0925 12:22:18.881912 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:18.881913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:18.881925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:18.881974 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:18.882610 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.883347 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.884262 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:18.884455 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:18.884520 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:18.887230 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:60029: BlockingRecv error: recv got EOF from 127.1.47.130:60029 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:19.020340 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:19.015550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:56839)
0925 12:22:19.015721 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:19.015732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:19.015802 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:19.015970 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:19.015971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:19.016160 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:19.016206 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.016208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.016876 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.016877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.017425 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.017427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.017807 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.017808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.017846 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.018343 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:19.018347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:19.020233 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
W20260925 12:22:19.020375 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:19.015661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:56839 (local address 127.1.47.190:46377)
0925 12:22:19.015740 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:19.015742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:19.015777 (+ 35us) server_negotiation.cc:407] Connection header received
0925 12:22:19.015887 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:19.015888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:19.015902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:19.015954 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:19.016273 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.016806 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.017517 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.017730 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.017794 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.020309 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:56839: BlockingRecv error: recv got EOF from 127.1.47.131:56839 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:19.776335 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:19.771583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:50213)
0925 12:22:19.771731 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:19.771741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:19.771788 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:19.771973 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:19.771974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:19.772147 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:19.772194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.772195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.772843 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.772844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.773396 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.773399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.773778 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.773779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.773822 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.774324 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:19.774328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:19.776267 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
W20260925 12:22:19.776420 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:19.771695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:50213 (local address 127.1.47.190:46377)
0925 12:22:19.771823 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:19.771825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:19.771841 (+ 16us) server_negotiation.cc:407] Connection header received
0925 12:22:19.771882 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:19.771882 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:19.771892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:19.771956 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:19.772260 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.772766 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.773493 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.773704 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.773765 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.776365 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:50213: BlockingRecv error: recv got EOF from 127.1.47.129:50213 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:19 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:19.892992 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:19.887904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:54509)
0925 12:22:19.888062 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:19.888077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:19.888139 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:19.888319 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:19.888320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:19.888488 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:19.888535 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.888537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.889537 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.889538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.890073 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:19.890075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.890410 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.890411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:19.890452 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.890887 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:19.890890 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:19.892905 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
W20260925 12:22:19.893128 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:19.887996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:54509 (local address 127.1.47.190:46377)
0925 12:22:19.888070 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:19.888079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:19.888140 (+ 61us) server_negotiation.cc:407] Connection header received
0925 12:22:19.888223 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:19.888224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:19.888234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:19.888299 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:19.888627 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.889465 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.890159 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:19.890344 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:19.890401 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:19.893048 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:54509: BlockingRecv error: recv got EOF from 127.1.47.130:54509 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:20.025691 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:20.020835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:42787)
0925 12:22:20.021014 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:20.021027 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:20.021142 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:20.021322 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:20.021324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:20.021576 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:20.021640 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.021642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.022395 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.022396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.022940 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.022942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.023255 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.023256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.023295 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.023747 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:20.023751 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:20.025610 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
W20260925 12:22:20.025856 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:20.020944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:42787 (local address 127.1.47.190:46377)
0925 12:22:20.021018 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:20.021020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:20.021129 (+ 109us) server_negotiation.cc:407] Connection header received
0925 12:22:20.021202 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:20.021203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:20.021218 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:20.021277 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:20.021757 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.022310 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.023016 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.023191 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.023242 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.025769 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:42787: BlockingRecv error: recv got EOF from 127.1.47.131:42787 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:20.781855 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:20.776956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:33509)
0925 12:22:20.777149 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:20.777163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:20.777238 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:20.777443 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:20.777445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:20.777678 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:20.777740 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.777742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.778458 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.778459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.778992 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.778993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.779385 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.779386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.779426 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.779985 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:20.779990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:20.781798 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
W20260925 12:22:20.781998 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:20.777118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33509 (local address 127.1.47.190:46377)
0925 12:22:20.777257 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:20.777259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:20.777269 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:20.777302 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:20.777303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:20.777317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:20.777422 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:20.777826 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.778373 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.779083 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.779318 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.779377 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.781904 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:33509: BlockingRecv error: recv got EOF from 127.1.47.129:33509 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:20 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:20.898802 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:20.893527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:54851)
0925 12:22:20.893709 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:20.893724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:20.893800 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:20.893984 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:20.893985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:20.894270 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:20.894326 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.894329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.894978 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.894978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.895546 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:20.895547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.896126 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.896127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:20.896181 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.896653 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:20.896658 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:20.898709 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
W20260925 12:22:20.898864 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:20.893705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:54851 (local address 127.1.47.190:46377)
0925 12:22:20.893776 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:20.893777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:20.893787 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:22:20.893891 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:20.893893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:20.893907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:20.893966 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:20.894406 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.894900 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.895659 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:20.895983 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:20.896065 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:20.898803 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:54851: BlockingRecv error: recv got EOF from 127.1.47.130:54851 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:21.031447 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:21.026167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:40849)
0925 12:22:21.026353 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:21.026365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:21.026466 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:21.026653 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:21.026654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:21.026851 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:21.026896 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.026898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.027582 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.027584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.028201 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.028204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.028567 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.028568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.028632 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.029119 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:21.029123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:21.031320 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":80,"threads_started":1}
W20260925 12:22:21.031510 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:21.026245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:40849 (local address 127.1.47.190:46377)
0925 12:22:21.026339 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:21.026342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:21.026443 (+ 101us) server_negotiation.cc:407] Connection header received
0925 12:22:21.026538 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:21.026539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:21.026553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:21.026641 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:21.026977 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.027490 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.028287 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.028480 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.028556 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.031440 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:40849: BlockingRecv error: recv got EOF from 127.1.47.131:40849 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:21.787400 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:21.782369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:46489)
0925 12:22:21.782653 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:21.782665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:21.782720 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:21.782917 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:21.782918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:21.783098 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:21.783173 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.783174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.783812 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.783813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.784410 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.784412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.784740 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.784740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.784780 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.785311 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:21.785316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:21.787322 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":192,"threads_started":1}
W20260925 12:22:21.787533 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:21.782554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:46489 (local address 127.1.47.190:46377)
0925 12:22:21.782694 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:21.782696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:21.782705 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:21.782784 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:21.782784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:21.782798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:21.782862 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:21.783240 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.783732 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.784477 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.784672 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.784730 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.787417 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:46489: BlockingRecv error: recv got EOF from 127.1.47.129:46489 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:21 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:21.904150 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:21.899358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:42241)
0925 12:22:21.899485 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:21.899498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:21.899570 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:21.899701 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:21.899701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:21.899913 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:21.899962 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.899964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.900577 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.900578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.901098 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:21.901100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.901445 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.901446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:21.901508 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.902112 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:21.902117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:21.904088 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
W20260925 12:22:21.904273 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:21.899397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:42241 (local address 127.1.47.190:46377)
0925 12:22:21.899450 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:21.899452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:21.899575 (+ 123us) server_negotiation.cc:407] Connection header received
0925 12:22:21.899624 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:21.899625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:21.899636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:21.899676 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:21.900016 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.900508 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.901175 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:21.901355 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:21.901412 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:21.904193 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:42241: BlockingRecv error: recv got EOF from 127.1.47.130:42241 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.131@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:22.036672 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:22.031965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.131:54161)
0925 12:22:22.032112 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:22.032160 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:22.032218 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:22.032350 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:22.032351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:22.032517 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:22.032563 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.032565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.033194 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.033194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.033726 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.033728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.034075 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.034076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.034120 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.034750 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:22.034754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:22.036595 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
W20260925 12:22:22.036779 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:22.032051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54161 (local address 127.1.47.190:46377)
0925 12:22:22.032100 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:22.032102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:22.032221 (+ 119us) server_negotiation.cc:407] Connection header received
0925 12:22:22.032274 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:22.032275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:22.032288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:22.032329 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:22.032624 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.033118 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.033802 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.033983 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.034034 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.036699 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.131:54161: BlockingRecv error: recv got EOF from 127.1.47.131:54161 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.129@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:22.797518 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:22.788039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.129:47723)
0925 12:22:22.788219 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:22.788233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:22.788313 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:22.788476 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:22.788477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:22.788641 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:22.788691 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.788693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.789508 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.789509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.790093 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.790095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.790488 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.790489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.790533 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.795379 (+ 4846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:22.795385 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:22.797446 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":55,"threads_started":1}
W20260925 12:22:22.797739 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:22.788147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:47723 (local address 127.1.47.190:46377)
0925 12:22:22.788302 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:22.788303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:22.788314 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:22:22.788363 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:22.788363 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:22.788375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:22.788435 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:22.788810 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.789353 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.790194 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.790394 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.790464 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.797652 (+ 7188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.129:47723: BlockingRecv error: recv got EOF from 127.1.47.129:47723 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 12:22:22 dist-test-slave-l9ww krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.47.130@KRBTEST.COM for oryx/127.1.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 12:22:22.909608 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:22.904880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46377 (local address 127.1.47.130:42735)
0925 12:22:22.905020 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:22.905033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:22.905091 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:22.905265 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:22.905266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:22.905429 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:22.905475 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.905476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.906183 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.906183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.906697 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:22.906698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.907098 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.907099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:22.907141 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.907602 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:22.907606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:22.909492 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.47.190:46377: Server oryx/127.1.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
W20260925 12:22:22.909651 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:22.904963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:42735 (local address 127.1.47.190:46377)
0925 12:22:22.905102 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:22.905105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:22.905151 (+ 46us) server_negotiation.cc:407] Connection header received
0925 12:22:22.905185 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:22.905185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:22.905196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:22.905247 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:22.905581 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.906111 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.906800 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:22.907014 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:22.907094 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:22.909578 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.47.130:42735: BlockingRecv error: recv got EOF from 127.1.47.130:42735 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260925 12:22:22.944458 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9953
I20260925 12:22:22.949143 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10105
I20260925 12:22:22.953403 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10237
I20260925 12:22:22.957937 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 9882
2026-09-25T12:22:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17589 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T12:22:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:22Z Disabled control of system clock
I20260925 12:22:22.986917 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37125
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 12:22:23.069222 10432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.069432 10432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.069453 10432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.070591 10432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:23.070629 10432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.070642 10432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.070654 10432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:23.070667 10432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:23.071996 10432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.072211 10432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.072463 10432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.077816 10438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.077847 10437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.077816 10440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.077858 10432 server_base.cc:1034] running on GCE node
I20260925 12:22:23.078372 10432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.078637 10432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.079780 10432 hybrid_clock.cc:648] HybridClock initialized: now 1790338943079751 us; error 42 us; skew 500 ppm
I20260925 12:22:23.081287 10432 webserver.cc:460] Webserver started at http://127.1.47.190:35247/ using document root <none> and password file <none>
I20260925 12:22:23.081581 10432 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.081636 10432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.081792 10432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.082942 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "e4e98592600846b8a91b531e32ac63e3"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "92ae89b66a5fb470698576b041c89ed9"
server_key_iv: "4585538379e84b9f04d1d8f8151859c3"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.083307 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "e4e98592600846b8a91b531e32ac63e3"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "92ae89b66a5fb470698576b041c89ed9"
server_key_iv: "4585538379e84b9f04d1d8f8151859c3"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.084765 10432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:22:23.085525 10446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.085726 10432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:23.085819 10432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "e4e98592600846b8a91b531e32ac63e3"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "92ae89b66a5fb470698576b041c89ed9"
server_key_iv: "4585538379e84b9f04d1d8f8151859c3"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.085878 10432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.097345 10432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.097635 10432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.097755 10432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.102006 10432 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37125
I20260925 12:22:23.102087 10498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37125 every 8 connection(s)
I20260925 12:22:23.102463 10432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:23.103286 10499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:23.105980 10499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: Bootstrap starting.
I20260925 12:22:23.106635 10499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:23.106930 10499 log.cc:824] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:23.107656 10499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: No bootstrap required, opened a new log
I20260925 12:22:23.109350 10499 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } }
I20260925 12:22:23.109516 10499 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:23.109563 10499 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e98592600846b8a91b531e32ac63e3, State: Initialized, Role: FOLLOWER
I20260925 12:22:23.109723 10499 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } }
I20260925 12:22:23.109807 10499 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:23.109849 10499 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:23.109920 10499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:23.110466 10499 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } }
I20260925 12:22:23.110584 10499 leader_election.cc:302] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4e98592600846b8a91b531e32ac63e3; no voters:
I20260925 12:22:23.110778 10499 leader_election.cc:288] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:22:23.110831 10502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:23.111040 10499 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:23.111127 10502 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [term 1 LEADER]: Becoming Leader. State: Replica: e4e98592600846b8a91b531e32ac63e3, State: Running, Role: LEADER
I20260925 12:22:23.111301 10502 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } }
I20260925 12:22:23.111613 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10432
I20260925 12:22:23.111702 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:22:23.111963 1214 external_mini_cluster.cc:1428] Setting key b884a39c40759e5a43af5c9a6be2b4f3
I20260925 12:22:23.111944 10503 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4e98592600846b8a91b531e32ac63e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } } }
I20260925 12:22:23.112032 10503 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:23.112722 10504 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4e98592600846b8a91b531e32ac63e3. Latest consensus state: current_term: 1 leader_uuid: "e4e98592600846b8a91b531e32ac63e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e98592600846b8a91b531e32ac63e3" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37125 } } }
I20260925 12:22:23.112946 10504 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3 [sys.catalog]: This master's current role is: LEADER
W20260925 12:22:23.113008 10519 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:22:23.113067 10519 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:22:23.113581 10511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:23.113813 10511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:23.115294 10511 catalog_manager.cc:1347] Generated new cluster ID: cc73a776240c4c2bbe0043110b9fce16
I20260925 12:22:23.115355 10511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:22:23.118701 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.112640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.1.47.190:37125)
0925 12:22:23.113109 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.113121 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.113158 (+ 37us) server_negotiation.cc:407] Connection header received
0925 12:22:23.113276 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.113281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.113340 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.113465 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.113742 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.115092 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.116255 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.116957 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.117097 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.117893 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.117897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.117897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.118123 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.118168 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.118184 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.118257 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.118361 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.118447 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":80,"threads_started":1}
I20260925 12:22:23.122599 10511 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:22:23.123232 10511 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:23.132570 10511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4e98592600846b8a91b531e32ac63e3: Generated new TSK 0
I20260925 12:22:23.132823 10511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:22:23.141180 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--builtin_ntp_servers=127.1.47.148:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 12:22:23.240100 10523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.240312 10523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.240334 10523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.241544 10523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.241587 10523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.241612 10523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:23.242945 10523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.243187 10523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.243501 10523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.249366 10529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.249366 10528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.249581 10523 server_base.cc:1034] running on GCE node
W20260925 12:22:23.249749 10531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.249989 10523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.250310 10523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.251513 10523 hybrid_clock.cc:648] HybridClock initialized: now 1790338943251498 us; error 34 us; skew 500 ppm
I20260925 12:22:23.253201 10523 webserver.cc:460] Webserver started at http://127.1.47.129:43159/ using document root <none> and password file <none>
I20260925 12:22:23.253480 10523 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.253566 10523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.253732 10523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.254949 10523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "6ce6396500004bc49aea90cbfa08f768"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "a9813562b7be70846ddd87fda1068143"
server_key_iv: "b0e1d6549b37991734a08575dbaec5aa"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.255394 10523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "6ce6396500004bc49aea90cbfa08f768"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "a9813562b7be70846ddd87fda1068143"
server_key_iv: "b0e1d6549b37991734a08575dbaec5aa"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.257259 10523 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260925 12:22:23.258247 10537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.258477 10523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.258595 10523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "6ce6396500004bc49aea90cbfa08f768"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "a9813562b7be70846ddd87fda1068143"
server_key_iv: "b0e1d6549b37991734a08575dbaec5aa"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.258680 10523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.274041 10523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.274377 10523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.274540 10523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.274782 10523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:23.275136 10523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:23.275192 10523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.275234 10523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:23.275262 10523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.281207 10523 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38613
I20260925 12:22:23.281286 10650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38613 every 8 connection(s)
I20260925 12:22:23.281682 10523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:22:23.286957 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10523
I20260925 12:22:23.287081 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:22:23.287122 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.282478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:48023 (local address 127.1.47.190:37125)
0925 12:22:23.282522 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.282524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.282782 (+ 258us) server_negotiation.cc:407] Connection header received
0925 12:22:23.283323 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.283324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.283339 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.283417 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.284134 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.284634 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.285651 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.285844 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.285902 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.286780 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.286781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.286781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.286786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.286799 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.286803 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.286904 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.287007 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.287039 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:22:23.287297 1214 external_mini_cluster.cc:1428] Setting key 83ab1f489d945aae47f7add78b2cab69
I20260925 12:22:23.287384 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.282439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37125 (local address 127.1.47.129:48023)
0925 12:22:23.282642 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:23.282683 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:23.283207 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:23.283522 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:23.283526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:23.283535 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:23.284052 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.284056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.284739 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.284739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.285556 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.285559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.285924 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.285924 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.286038 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.286628 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:23.286629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:23.286659 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:23.286684 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:23.286908 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:23.286915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:23.286918 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:23.287010 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 12:22:23.287114 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260925 12:22:23.287971 10651 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37125
I20260925 12:22:23.288091 10651 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:23.288259 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--builtin_ntp_servers=127.1.47.148:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 12:22:23.288281 10651 heartbeater.cc:511] Master 127.1.47.190:37125 requested a full tablet report, sending...
I20260925 12:22:23.288847 10463 ts_manager.cc:194] Registered new tserver with Master: 6ce6396500004bc49aea90cbfa08f768 (127.1.47.129:38613)
I20260925 12:22:23.289479 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.129:48023
W20260925 12:22:23.370424 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.370642 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.370666 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.371954 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.372001 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.372028 10654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:23.373492 10654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.373761 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.374086 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.379129 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.379097 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.379159 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.379496 10654 server_base.cc:1034] running on GCE node
I20260925 12:22:23.379712 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.379998 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.381148 10654 hybrid_clock.cc:648] HybridClock initialized: now 1790338943381136 us; error 38 us; skew 500 ppm
I20260925 12:22:23.382500 10654 webserver.cc:460] Webserver started at http://127.1.47.130:39023/ using document root <none> and password file <none>
I20260925 12:22:23.382815 10654 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.382884 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.383028 10654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.384202 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "5db84090d0704da1978e0e8bb75a16d9"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "8a33339c2ceff3f216b003c1194bcdb3"
server_key_iv: "9eee3cb8b4b0225117a62ef8d725a2ca"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.384555 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "5db84090d0704da1978e0e8bb75a16d9"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "8a33339c2ceff3f216b003c1194bcdb3"
server_key_iv: "9eee3cb8b4b0225117a62ef8d725a2ca"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.385861 10654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.386626 10668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.386813 10654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.386909 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "5db84090d0704da1978e0e8bb75a16d9"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "8a33339c2ceff3f216b003c1194bcdb3"
server_key_iv: "9eee3cb8b4b0225117a62ef8d725a2ca"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.386996 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.410127 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.410544 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.410749 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.411046 10654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:23.411510 10654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:23.411577 10654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.411631 10654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:23.411660 10654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.418592 10654 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:42601
I20260925 12:22:23.418633 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:42601 every 8 connection(s)
I20260925 12:22:23.419024 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:22:23.423451 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10654
I20260925 12:22:23.423534 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:22:23.423765 1214 external_mini_cluster.cc:1428] Setting key a01919b606c5d9d83c9a29eb3361e799
I20260925 12:22:23.424809 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--builtin_ntp_servers=127.1.47.148:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 12:22:23.424924 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.419979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:39191 (local address 127.1.47.190:37125)
0925 12:22:23.420093 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.420095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.420318 (+ 223us) server_negotiation.cc:407] Connection header received
0925 12:22:23.420878 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.420879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.420891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.420957 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.421706 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.422195 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.423294 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.423480 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.423588 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.424651 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.424652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.424652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.424656 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.424670 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.424674 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.424715 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.424797 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.424821 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 12:22:23.425199 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.419940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37125 (local address 127.1.47.130:39191)
0925 12:22:23.420155 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:23.420200 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:23.420738 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:23.421033 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:23.421037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:23.421045 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:23.421606 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.421611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.422306 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.422307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.423163 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.423166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.423612 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.423613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.423791 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.424491 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:23.424492 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:23.424535 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:23.424566 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:23.424718 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:23.424722 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:23.424726 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:23.424797 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 12:22:23.424920 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260925 12:22:23.425800 10782 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37125
I20260925 12:22:23.425907 10782 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:23.426146 10782 heartbeater.cc:511] Master 127.1.47.190:37125 requested a full tablet report, sending...
I20260925 12:22:23.426582 10463 ts_manager.cc:194] Registered new tserver with Master: 5db84090d0704da1978e0e8bb75a16d9 (127.1.47.130:42601)
I20260925 12:22:23.426975 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.130:39191
W20260925 12:22:23.506248 10785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.506508 10785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.506546 10785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.507704 10785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.507773 10785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.507812 10785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:23.509181 10785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.509457 10785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.509766 10785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.514914 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.514904 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.514905 10793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.515111 10785 server_base.cc:1034] running on GCE node
I20260925 12:22:23.515377 10785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.515641 10785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.516850 10785 hybrid_clock.cc:648] HybridClock initialized: now 1790338943516814 us; error 48 us; skew 500 ppm
I20260925 12:22:23.518308 10785 webserver.cc:460] Webserver started at http://127.1.47.131:34883/ using document root <none> and password file <none>
I20260925 12:22:23.518564 10785 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.518635 10785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.518793 10785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.520021 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "f56ff1f583044178bdbbddcb440542db"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b960ff82afefcf547db038904c78d1b0"
server_key_iv: "1e16c02ba5f12798a2050308cf4c633f"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.520509 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "f56ff1f583044178bdbbddcb440542db"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b960ff82afefcf547db038904c78d1b0"
server_key_iv: "1e16c02ba5f12798a2050308cf4c633f"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.521926 10785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:22:23.522773 10799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.522979 10785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:23.523088 10785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "f56ff1f583044178bdbbddcb440542db"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b960ff82afefcf547db038904c78d1b0"
server_key_iv: "1e16c02ba5f12798a2050308cf4c633f"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.523175 10785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.537562 10785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.537909 10785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.538069 10785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.538314 10785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:23.538668 10785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:23.538731 10785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.538774 10785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:23.538802 10785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.544747 10785 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:43507
I20260925 12:22:23.544793 10912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:43507 every 8 connection(s)
I20260925 12:22:23.545136 10785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:22:23.550046 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10785
I20260925 12:22:23.550153 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:22:23.550412 1214 external_mini_cluster.cc:1428] Setting key 934ad5a885c5e57e579a12ba6652fb9a
I20260925 12:22:23.550705 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.545891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:32921 (local address 127.1.47.190:37125)
0925 12:22:23.545982 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.545984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.546203 (+ 219us) server_negotiation.cc:407] Connection header received
0925 12:22:23.546763 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.546764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.546777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.546824 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.547514 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.548045 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.549078 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.549282 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.549371 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.550329 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.550330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.550331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.550336 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.550352 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.550357 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.550414 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.550547 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.550592 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:22:23.550865 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.545858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37125 (local address 127.1.47.131:32921)
0925 12:22:23.546081 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:23.546116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:23.546644 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:23.546896 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:23.546900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:23.546910 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:23.547419 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.547423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.548178 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.548179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.548969 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:23.548971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:23.549377 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:23.549378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:23.549520 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:23.550172 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:23.550173 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:23.550213 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:23.550242 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:23.550430 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:23.550436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:23.550440 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:23.550519 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 12:22:23.550634 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260925 12:22:23.551316 10913 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37125
I20260925 12:22:23.551424 10913 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:23.551646 10913 heartbeater.cc:511] Master 127.1.47.190:37125 requested a full tablet report, sending...
I20260925 12:22:23.552047 10463 ts_manager.cc:194] Registered new tserver with Master: f56ff1f583044178bdbbddcb440542db (127.1.47.131:43507)
I20260925 12:22:23.552419 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.131:32921
I20260925 12:22:23.562767 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 12:22:23.564771 10520 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:23.564296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.1.47.190:37125)
0925 12:22:23.564383 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.564385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.564394 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:22:23.564448 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.564449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.564460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.564533 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.564690 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41814: BlockingRecv error: recv got EOF from 127.0.0.1:41814 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 12:22:23.564879 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10523
I20260925 12:22:23.569497 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10654
I20260925 12:22:23.573760 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10785
I20260925 12:22:23.578209 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10432
2026-09-25T12:22:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (618 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T12:22:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:23Z Disabled control of system clock
I20260925 12:22:23.604468 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46057
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 12:22:23.684736 10924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.684957 10924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.684978 10924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.686059 10924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:23.686093 10924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.686112 10924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.686125 10924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:23.686137 10924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:23.687393 10924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.687583 10924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.687808 10924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.692672 10929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.692683 10932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.692718 10930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.692965 10924 server_base.cc:1034] running on GCE node
I20260925 12:22:23.693185 10924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.693424 10924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.694558 10924 hybrid_clock.cc:648] HybridClock initialized: now 1790338943694545 us; error 31 us; skew 500 ppm
I20260925 12:22:23.695787 10924 webserver.cc:460] Webserver started at http://127.1.47.190:37291/ using document root <none> and password file <none>
I20260925 12:22:23.696038 10924 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.696085 10924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.696265 10924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.697340 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "345ae18d393d48f69959ccbc1a63dc1b"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b3e26ee1f207e1749ffd9d6c5d52e87c"
server_key_iv: "a7d6e758e5eccc05ed70be686969c006"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.697748 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "345ae18d393d48f69959ccbc1a63dc1b"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b3e26ee1f207e1749ffd9d6c5d52e87c"
server_key_iv: "a7d6e758e5eccc05ed70be686969c006"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.699239 10924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:22:23.700086 10938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.700266 10924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.700341 10924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "345ae18d393d48f69959ccbc1a63dc1b"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "b3e26ee1f207e1749ffd9d6c5d52e87c"
server_key_iv: "a7d6e758e5eccc05ed70be686969c006"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.700397 10924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.703753 10924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.704030 10924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.704129 10924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.707814 10924 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46057
I20260925 12:22:23.707870 10990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46057 every 8 connection(s)
I20260925 12:22:23.708231 10924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:23.708432 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 10924
I20260925 12:22:23.708523 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:22:23.708719 1214 external_mini_cluster.cc:1428] Setting key 99c844cbd82dcb5eb5d7b7467778c256
I20260925 12:22:23.709039 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:23.711299 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.709270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.1.47.190:46057)
0925 12:22:23.709481 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.709488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.709513 (+ 25us) server_negotiation.cc:407] Connection header received
0925 12:22:23.709937 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.709943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.709979 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.710098 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.710798 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.710801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.710802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.710805 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.710847 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.710857 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.710914 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.710999 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.711070 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260925 12:22:23.712555 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b: Bootstrap starting.
I20260925 12:22:23.713160 10991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:23.713434 10991 log.cc:824] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:23.714087 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b: No bootstrap required, opened a new log
I20260925 12:22:23.715459 10991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } }
I20260925 12:22:23.715653 10991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:23.715683 10991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345ae18d393d48f69959ccbc1a63dc1b, State: Initialized, Role: FOLLOWER
I20260925 12:22:23.715840 10991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } }
I20260925 12:22:23.715953 10991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:23.716006 10991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:23.716055 10991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:23.716564 10991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } }
I20260925 12:22:23.716681 10991 leader_election.cc:302] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345ae18d393d48f69959ccbc1a63dc1b; no voters:
I20260925 12:22:23.716840 10991 leader_election.cc:288] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:22:23.717062 10996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:23.717118 10991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:23.717245 10996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [term 1 LEADER]: Becoming Leader. State: Replica: 345ae18d393d48f69959ccbc1a63dc1b, State: Running, Role: LEADER
I20260925 12:22:23.717352 10996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } }
I20260925 12:22:23.717774 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345ae18d393d48f69959ccbc1a63dc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } } }
I20260925 12:22:23.717878 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:23.718041 10998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345ae18d393d48f69959ccbc1a63dc1b. Latest consensus state: current_term: 1 leader_uuid: "345ae18d393d48f69959ccbc1a63dc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ae18d393d48f69959ccbc1a63dc1b" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46057 } } }
I20260925 12:22:23.718269 10998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:23.718339 11005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:23.718883 11005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:23.720422 11005 catalog_manager.cc:1347] Generated new cluster ID: 751da37fca784247b930b4710ba2821a
I20260925 12:22:23.720475 11005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:22:23.735699 11005 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:22:23.735924 11005 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:23.741271 11005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345ae18d393d48f69959ccbc1a63dc1b: Generated new TSK 0
I20260925 12:22:23.741459 11005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:22:23.746469 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--builtin_ntp_servers=127.1.47.148:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 12:22:23.829867 11015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.830127 11015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.830154 11015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.831610 11015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.831656 11015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.831687 11015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:23.833236 11015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.833511 11015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.833846 11015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.839962 11021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.839838 11023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.840163 11015 server_base.cc:1034] running on GCE node
W20260925 12:22:23.839881 11020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.840432 11015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.840718 11015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.841894 11015 hybrid_clock.cc:648] HybridClock initialized: now 1790338943841875 us; error 37 us; skew 500 ppm
I20260925 12:22:23.843118 11015 webserver.cc:460] Webserver started at http://127.1.47.129:41987/ using document root <none> and password file <none>
I20260925 12:22:23.843403 11015 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.843479 11015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.843633 11015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.844828 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "802b5b384b53445582874cf4dcd49c45"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "57f440d34d0b1d721816f86d4558fe24"
server_key_iv: "3b9ceddf748471f7ed0b0f51549b5cab"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.845227 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "802b5b384b53445582874cf4dcd49c45"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "57f440d34d0b1d721816f86d4558fe24"
server_key_iv: "3b9ceddf748471f7ed0b0f51549b5cab"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.846657 11015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:22:23.847498 11029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.847692 11015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.847806 11015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "802b5b384b53445582874cf4dcd49c45"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "57f440d34d0b1d721816f86d4558fe24"
server_key_iv: "3b9ceddf748471f7ed0b0f51549b5cab"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.847929 11015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.851315 11015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.851616 11015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.851766 11015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.852025 11015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:23.852355 11015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:23.852414 11015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.852448 11015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:23.852475 11015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.858476 11015 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:44069
I20260925 12:22:23.858522 11142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:44069 every 8 connection(s)
I20260925 12:22:23.858865 11015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:22:23.861429 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11015
I20260925 12:22:23.861537 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:22:23.861790 1214 external_mini_cluster.cc:1428] Setting key 7dde6af967213758323cd2476f72d40e
I20260925 12:22:23.862041 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.859707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33979 (local address 127.1.47.190:46057)
0925 12:22:23.859776 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.859778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.859973 (+ 195us) server_negotiation.cc:407] Connection header received
0925 12:22:23.860448 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.860449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.860454 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.860530 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.861719 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.861722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.861722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.861728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.861746 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.861752 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.861830 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.861941 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.861959 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:22:23.862242 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.859623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46057 (local address 127.1.47.129:33979)
0925 12:22:23.859827 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:23.859893 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:23.860359 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:23.860586 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:23.860589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:23.860619 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:23.861513 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:23.861513 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:23.861564 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:23.861611 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:23.861826 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:23.861829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:23.861831 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:22:23.861930 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 12:22:23.862005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260925 12:22:23.862783 11143 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46057
I20260925 12:22:23.862880 11143 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:23.862955 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--builtin_ntp_servers=127.1.47.148:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 12:22:23.863095 11143 heartbeater.cc:511] Master 127.1.47.190:46057 requested a full tablet report, sending...
I20260925 12:22:23.863657 10955 ts_manager.cc:194] Registered new tserver with Master: 802b5b384b53445582874cf4dcd49c45 (127.1.47.129:44069)
W20260925 12:22:23.945210 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:23.945433 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:23.945457 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:23.946668 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:23.946714 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:23.946743 11146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:23.948287 11146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:23.948539 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:23.948832 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:23.953936 11154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.953933 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:23.953940 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:23.954291 11146 server_base.cc:1034] running on GCE node
I20260925 12:22:23.954541 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:23.954792 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:23.955928 11146 hybrid_clock.cc:648] HybridClock initialized: now 1790338943955911 us; error 35 us; skew 500 ppm
I20260925 12:22:23.957362 11146 webserver.cc:460] Webserver started at http://127.1.47.130:33231/ using document root <none> and password file <none>
I20260925 12:22:23.957646 11146 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:23.957724 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:23.957880 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:23.959467 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "ff947cc6fee3492686bbaee7c6f3e46d"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "14306c24f8ff996f380ca46aa2c2648e"
server_key_iv: "f1b2db4a2152bf36824bd2980a14647b"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.959988 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "ff947cc6fee3492686bbaee7c6f3e46d"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "14306c24f8ff996f380ca46aa2c2648e"
server_key_iv: "f1b2db4a2152bf36824bd2980a14647b"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.961887 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 12:22:23.962859 11160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.963054 11146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:23.963169 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "ff947cc6fee3492686bbaee7c6f3e46d"
format_stamp: "Formatted at 2026-09-25 12:22:23 on dist-test-slave-l9ww"
server_key: "14306c24f8ff996f380ca46aa2c2648e"
server_key_iv: "f1b2db4a2152bf36824bd2980a14647b"
server_key_version: "encryptionkey@0"
I20260925 12:22:23.963255 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:23.967350 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:23.967576 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:23.967715 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:23.967949 11146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:23.968273 11146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:23.968324 11146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.968365 11146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:23.968391 11146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:23.974126 11146 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:38211
I20260925 12:22:23.974202 11273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:38211 every 8 connection(s)
I20260925 12:22:23.974591 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:22:23.977880 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11146
I20260925 12:22:23.977998 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:22:23.978200 1214 external_mini_cluster.cc:1428] Setting key 3e1a460ed2d5b34512268e4088e84ea4
I20260925 12:22:23.978565 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.975436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:47425 (local address 127.1.47.190:46057)
0925 12:22:23.975523 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:23.975525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:23.975916 (+ 391us) server_negotiation.cc:407] Connection header received
0925 12:22:23.976610 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:23.976611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:23.976618 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:23.976687 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:23.978272 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:23.978274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:23.978274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:23.978280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:23.978295 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:23.978300 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:23.978353 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:23.978473 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 12:22:23.978489 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:22:23.978766 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:23.975459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46057 (local address 127.1.47.130:47425)
0925 12:22:23.975706 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:23.975781 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:23.976497 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:23.976839 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:23.976844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:23.976871 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:23.978079 (+ 1208us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:23.978080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:23.978124 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:23.978161 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:23.978376 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:23.978381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:23.978385 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:22:23.978470 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 12:22:23.978540 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260925 12:22:23.979159 11274 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46057
I20260925 12:22:23.979264 11274 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:23.979334 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--builtin_ntp_servers=127.1.47.148:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 12:22:23.979426 11274 heartbeater.cc:511] Master 127.1.47.190:46057 requested a full tablet report, sending...
I20260925 12:22:23.979764 10955 ts_manager.cc:194] Registered new tserver with Master: ff947cc6fee3492686bbaee7c6f3e46d (127.1.47.130:38211)
W20260925 12:22:24.063434 11277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:24.063705 11277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:24.063745 11277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:24.065121 11277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:24.065191 11277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:24.065238 11277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:24.066699 11277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:24.066990 11277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:24.067308 11277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:24.072638 11283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:24.072791 11277 server_base.cc:1034] running on GCE node
W20260925 12:22:24.072649 11285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:24.072872 11282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:24.073081 11277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:24.073365 11277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:24.074529 11277 hybrid_clock.cc:648] HybridClock initialized: now 1790338944074494 us; error 62 us; skew 500 ppm
I20260925 12:22:24.076004 11277 webserver.cc:460] Webserver started at http://127.1.47.131:43161/ using document root <none> and password file <none>
I20260925 12:22:24.076213 11277 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:24.076256 11277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:24.076349 11277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:24.077397 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "ceb536e4b4e2430e9ffd54a604236180"
format_stamp: "Formatted at 2026-09-25 12:22:24 on dist-test-slave-l9ww"
server_key: "5d1676b1ac2bf62ceb780b99d41c3624"
server_key_iv: "c1cc747790ce903faf207d14187cbf5a"
server_key_version: "encryptionkey@0"
I20260925 12:22:24.077705 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "ceb536e4b4e2430e9ffd54a604236180"
format_stamp: "Formatted at 2026-09-25 12:22:24 on dist-test-slave-l9ww"
server_key: "5d1676b1ac2bf62ceb780b99d41c3624"
server_key_iv: "c1cc747790ce903faf207d14187cbf5a"
server_key_version: "encryptionkey@0"
I20260925 12:22:24.078997 11277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:22:24.079773 11291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:24.079988 11277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:24.080070 11277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "ceb536e4b4e2430e9ffd54a604236180"
format_stamp: "Formatted at 2026-09-25 12:22:24 on dist-test-slave-l9ww"
server_key: "5d1676b1ac2bf62ceb780b99d41c3624"
server_key_iv: "c1cc747790ce903faf207d14187cbf5a"
server_key_version: "encryptionkey@0"
I20260925 12:22:24.080124 11277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:24.083398 11277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:24.083612 11277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:24.083719 11277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:24.083931 11277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:24.084240 11277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:24.084275 11277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:24.084295 11277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:24.084307 11277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:24.089790 11277 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38863
I20260925 12:22:24.089872 11404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38863 every 8 connection(s)
I20260925 12:22:24.090207 11277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:22:24.093606 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:24.091015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:36883 (local address 127.1.47.190:46057)
0925 12:22:24.091085 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:24.091088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:24.091406 (+ 318us) server_negotiation.cc:407] Connection header received
0925 12:22:24.091923 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:24.091924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:24.091932 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:24.092012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:24.093277 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:24.093278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:24.093278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:22:24.093283 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:24.093296 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:22:24.093300 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:24.093361 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:24.093528 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 12:22:24.093543 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:22:24.093899 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:24.091017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46057 (local address 127.1.47.131:36883)
0925 12:22:24.091268 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:24.091305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:24.091783 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:24.092072 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:24.092076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:24.092085 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:24.093113 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:22:24.093114 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:24.093143 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:22:24.093173 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:24.093425 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:24.093431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:24.093433 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:22:24.093513 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:22:24.093567 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260925 12:22:24.094463 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11277
I20260925 12:22:24.094563 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:22:24.094547 11405 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46057
I20260925 12:22:24.094677 11405 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:24.094802 1214 external_mini_cluster.cc:1428] Setting key 773c5c9b8601dc06c15221b3fe361c0e
I20260925 12:22:24.094828 11405 heartbeater.cc:511] Master 127.1.47.190:46057 requested a full tablet report, sending...
I20260925 12:22:24.095250 10955 ts_manager.cc:194] Registered new tserver with Master: ceb536e4b4e2430e9ffd54a604236180 (127.1.47.131:38863)
I20260925 12:22:24.096027 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 12:22:24.097736 10994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:22:24.097330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.1.47.190:46057)
0925 12:22:24.097372 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:24.097374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:24.097485 (+ 111us) server_negotiation.cc:407] Connection header received
0925 12:22:24.097525 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:24.097525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:24.097529 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:24.097586 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:24.097679 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40126: BlockingRecv error: recv got EOF from 127.0.0.1:40126 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:22:24.097795 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11015
I20260925 12:22:24.102383 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11146
I20260925 12:22:24.107092 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11277
I20260925 12:22:24.111765 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 10924
2026-09-25T12:22:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (534 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:22:24 dist-test-slave-l9ww krb5kdc[11415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:22:24 dist-test-slave-l9ww krb5kdc[11415](info): set up 2 sockets
Sep 25 12:22:24 dist-test-slave-l9ww krb5kdc[11415](info): commencing operation
krb5kdc: starting...
W20260925 12:22:25.152933 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:22:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:22:25.276978 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41637
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:25.358227 11428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:25.358495 11428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:25.358536 11428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:25.359675 11428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:22:25.359747 11428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:25.359772 11428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:25.359799 11428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:22:25.359836 11428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:22:25.361299 11428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:25.361598 11428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:25.361826 11428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 12:22:25.366848 11428 server_base.cc:1034] running on GCE node
W20260925 12:22:25.366806 11436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.366780 11433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.366788 11434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.367355 11428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:25.367664 11428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:25.368816 11428 hybrid_clock.cc:648] HybridClock initialized: now 1790338945368799 us; error 33 us; skew 500 ppm
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:25.371362 11428 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:22:25.371891 11428 webserver.cc:460] Webserver started at http://127.1.47.190:43327/ using document root <none> and password file <none>
I20260925 12:22:25.372143 11428 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:25.372188 11428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:25.372336 11428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:25.373454 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "cc2cd144c4994399a3d4ae13a7a9ee25"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9ce9e2c5d2ba3c3935f2876936b58c0c"
server_key_iv: "8354f503bd0e994195ff1684ab494f87"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.373823 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "cc2cd144c4994399a3d4ae13a7a9ee25"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9ce9e2c5d2ba3c3935f2876936b58c0c"
server_key_iv: "8354f503bd0e994195ff1684ab494f87"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.375216 11428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:22:25.376070 11443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.376291 11428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:25.376392 11428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "cc2cd144c4994399a3d4ae13a7a9ee25"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9ce9e2c5d2ba3c3935f2876936b58c0c"
server_key_iv: "8354f503bd0e994195ff1684ab494f87"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.376472 11428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:25.385720 11428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:25.388229 11428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:25.388350 11428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:25.392570 11428 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:41637
I20260925 12:22:25.392653 11495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:41637 every 8 connection(s)
I20260925 12:22:25.393009 11428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:22:25.393975 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:25.396377 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: Bootstrap starting.
I20260925 12:22:25.396943 11496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:25.397240 11496 log.cc:824] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:25.398000 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: No bootstrap required, opened a new log
I20260925 12:22:25.399561 11496 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } }
I20260925 12:22:25.399724 11496 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:25.399773 11496 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc2cd144c4994399a3d4ae13a7a9ee25, State: Initialized, Role: FOLLOWER
I20260925 12:22:25.399971 11496 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } }
I20260925 12:22:25.400087 11496 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:22:25.400132 11496 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:22:25.400183 11496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:25.400954 11496 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } }
I20260925 12:22:25.401091 11496 leader_election.cc:302] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc2cd144c4994399a3d4ae13a7a9ee25; no voters:
I20260925 12:22:25.401271 11496 leader_election.cc:288] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:22:25.401310 11499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:25.401521 11496 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:22:25.401536 11499 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [term 1 LEADER]: Becoming Leader. State: Replica: cc2cd144c4994399a3d4ae13a7a9ee25, State: Running, Role: LEADER
I20260925 12:22:25.401646 11499 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } }
I20260925 12:22:25.402051 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } } }
I20260925 12:22:25.402081 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11428
I20260925 12:22:25.402138 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:25.402179 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:22:25.402321 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2cd144c4994399a3d4ae13a7a9ee25. Latest consensus state: current_term: 1 leader_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2cd144c4994399a3d4ae13a7a9ee25" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41637 } } }
I20260925 12:22:25.402388 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25 [sys.catalog]: This master's current role is: LEADER
I20260925 12:22:25.402391 1214 external_mini_cluster.cc:1428] Setting key b6c3c8eff89016131fd8ad431c9fa626
I20260925 12:22:25.402522 11508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:22:25.403077 11508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:22:25.404842 11508 catalog_manager.cc:1347] Generated new cluster ID: b310498f76ca45ea882f89dbe9f98864
I20260925 12:22:25.404899 11508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:25.412194 11508 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:22:25.412791 11508 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:22:25.416157 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.403003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.1.47.190:41637)
0925 12:22:25.403965 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.403975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.404006 (+ 31us) server_negotiation.cc:407] Connection header received
0925 12:22:25.404123 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.404130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.404196 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.405113 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:25.405204 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.406557 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.407668 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.408414 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.408847 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.411431 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:25.411435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:25.411435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:25.411448 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:25.414065 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.414483 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.414486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.414486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.414631 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.414849 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.414849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.414850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.415121 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:25.415662 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.415750 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.415931 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":72,"threads_started":1}
I20260925 12:22:25.423009 11508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: Generated new TSK 0
I20260925 12:22:25.423204 11508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:22:25.450892 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:25.534005 11522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:25.534221 11522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:25.534245 11522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:25.535507 11522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:25.535547 11522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:25.535600 11522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:22:25.537112 11522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:25.537358 11522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:25.537618 11522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:25.542730 11530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.542790 11522 server_base.cc:1034] running on GCE node
W20260925 12:22:25.542742 11527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.542748 11528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.543161 11522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:25.543366 11522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:25.544534 11522 hybrid_clock.cc:648] HybridClock initialized: now 1790338945544524 us; error 30 us; skew 500 ppm
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:25.547266 11522 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:22:25.547910 11522 webserver.cc:460] Webserver started at http://127.1.47.129:42165/ using document root <none> and password file <none>
I20260925 12:22:25.548167 11522 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:25.548218 11522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:25.548370 11522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:25.549432 11522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "fad7b733d04a4bb2be8ef00778796914"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "2bc742a8dbbc6523d45a2738372e147b"
server_key_iv: "be0ae53e113f29a8e1a5d349556e7c81"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.549796 11522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "fad7b733d04a4bb2be8ef00778796914"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "2bc742a8dbbc6523d45a2738372e147b"
server_key_iv: "be0ae53e113f29a8e1a5d349556e7c81"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.551149 11522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:22:25.552016 11537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.552208 11522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:22:25.552300 11522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "fad7b733d04a4bb2be8ef00778796914"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "2bc742a8dbbc6523d45a2738372e147b"
server_key_iv: "be0ae53e113f29a8e1a5d349556e7c81"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.552390 11522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:25.571000 11522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:25.573253 11522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:25.573421 11522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:25.573666 11522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:25.574004 11522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:25.574060 11522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.574132 11522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:25.574162 11522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.580179 11522 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38393
I20260925 12:22:25.580253 11650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38393 every 8 connection(s)
I20260925 12:22:25.580580 11522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:22:25.585963 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11522
I20260925 12:22:25.586071 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:22:25.586248 1214 external_mini_cluster.cc:1428] Setting key 01ed6882f1964f09fe700d121d043e51
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:25.590601 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.581472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:35909 (local address 127.1.47.190:41637)
0925 12:22:25.581523 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.581525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.581708 (+ 183us) server_negotiation.cc:407] Connection header received
0925 12:22:25.582188 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.582189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.582197 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.582261 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:25.583213 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.583708 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.584819 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.585016 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.585076 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.587519 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:25.587520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:25.587521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:25.587530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:25.589057 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.589582 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.589583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.589583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.589654 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.589916 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.589917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.589917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.590036 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:25.590116 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.590417 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.590509 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:22:25.590817 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.581398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41637 (local address 127.1.47.129:35909)
0925 12:22:25.581568 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.581611 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.582105 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.582318 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.582321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.582908 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:25.583121 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.583127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.583802 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.583803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.584735 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.584737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.585116 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.585117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.585234 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.585818 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:25.585827 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:25.587427 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:25.589159 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.589169 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.589172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.589501 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.589725 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.589726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.589727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.589840 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.590117 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:25.590143 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:25.590285 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.590399 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.590566 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260925 12:22:25.591336 11651 heartbeater.cc:347] Connected to a master server at 127.1.47.190:41637
I20260925 12:22:25.591456 11651 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:25.591688 11651 heartbeater.cc:511] Master 127.1.47.190:41637 requested a full tablet report, sending...
I20260925 12:22:25.592275 11460 ts_manager.cc:194] Registered new tserver with Master: fad7b733d04a4bb2be8ef00778796914 (127.1.47.129:38393)
I20260925 12:22:25.592913 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:35909
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:22:25.611214 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:25.692453 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:25.692662 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:25.692683 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:25.693758 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:25.693792 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:25.693817 11656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:22:25.695163 11656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:25.695343 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:25.695593 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:25.700687 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.700773 11656 server_base.cc:1034] running on GCE node
W20260925 12:22:25.700672 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.700687 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.701108 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:25.701330 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:25.702493 11656 hybrid_clock.cc:648] HybridClock initialized: now 1790338945702446 us; error 53 us; skew 500 ppm
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:25.705320 11656 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:22:25.705844 11656 webserver.cc:460] Webserver started at http://127.1.47.130:44727/ using document root <none> and password file <none>
I20260925 12:22:25.706086 11656 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:25.706156 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:25.706311 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:25.707384 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "8b581742c4c34eaf8660e237fad6ad33"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9d25cf078710f460ee4933d86ea599ac"
server_key_iv: "44a37c34d5a6af9dc75c59fc19b1d357"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.707768 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "8b581742c4c34eaf8660e237fad6ad33"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9d25cf078710f460ee4933d86ea599ac"
server_key_iv: "44a37c34d5a6af9dc75c59fc19b1d357"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.709221 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:25.710060 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.710269 11656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:22:25.710372 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "8b581742c4c34eaf8660e237fad6ad33"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "9d25cf078710f460ee4933d86ea599ac"
server_key_iv: "44a37c34d5a6af9dc75c59fc19b1d357"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.710464 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:25.719733 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:25.722018 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:25.722206 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:25.722455 11656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:25.722781 11656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:25.722836 11656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.722872 11656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:25.722898 11656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.728868 11656 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:44937
I20260925 12:22:25.728914 11784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:44937 every 8 connection(s)
I20260925 12:22:25.729275 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:25.736224 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11656
I20260925 12:22:25.736326 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:22:25.736575 1214 external_mini_cluster.cc:1428] Setting key b70fe52dad3ade4ac46319f2448fb386
I20260925 12:22:25.739051 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.730145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:51527 (local address 127.1.47.190:41637)
0925 12:22:25.730237 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.730240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.730400 (+ 160us) server_negotiation.cc:407] Connection header received
0925 12:22:25.730842 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.730843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.730854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.730924 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:25.731908 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.732405 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.733473 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.733682 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.733751 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.736064 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:25.736066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:25.736066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:25.736074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:25.737656 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.738085 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.738087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.738087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.738149 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.738401 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.738402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.738402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.738524 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:25.738602 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.738880 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.738960 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 12:22:25.739283 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.730044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41637 (local address 127.1.47.130:51527)
0925 12:22:25.730207 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.730252 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.730742 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.730981 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.730984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.731568 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:25.731769 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.731773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.732515 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.732516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.733360 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.733362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.733781 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.733783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.733896 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.734415 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:25.734423 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:25.735912 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:25.737748 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.737756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.737759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.737986 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.738220 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.738221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.738221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.738331 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.738595 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:25.738601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:25.738743 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.738850 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.739016 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260925 12:22:25.739820 11785 heartbeater.cc:347] Connected to a master server at 127.1.47.190:41637
I20260925 12:22:25.739941 11785 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:25.740129 11785 heartbeater.cc:511] Master 127.1.47.190:41637 requested a full tablet report, sending...
I20260925 12:22:25.740532 11460 ts_manager.cc:194] Registered new tserver with Master: 8b581742c4c34eaf8660e237fad6ad33 (127.1.47.130:44937)
I20260925 12:22:25.740911 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:51527
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:22:25.762235 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:22:25.846487 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:22:25.846694 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:22:25.846717 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:22:25.847972 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:22:25.848016 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:22:25.848042 11790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:22:25.849490 11790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:22:25.849745 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:22:25.850003 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:22:25.855314 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.855358 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:22:25.855314 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:22:25.855613 11790 server_base.cc:1034] running on GCE node
I20260925 12:22:25.855952 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:22:25.856246 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:22:25.857406 11790 hybrid_clock.cc:648] HybridClock initialized: now 1790338945857393 us; error 35 us; skew 500 ppm
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:22:25.860107 11790 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:22:25.860546 11790 webserver.cc:460] Webserver started at http://127.1.47.131:42221/ using document root <none> and password file <none>
I20260925 12:22:25.860786 11790 fs_manager.cc:359] Metadata directory not provided
I20260925 12:22:25.860833 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:22:25.860983 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:22:25.862052 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "28c15f86af824b1291ee7aafbff15ccd"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "900c72687b9876281025809eb36be31b"
server_key_iv: "efd94d6fd2f0dcffeabbe36127d34b44"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.862414 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "28c15f86af824b1291ee7aafbff15ccd"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "900c72687b9876281025809eb36be31b"
server_key_iv: "efd94d6fd2f0dcffeabbe36127d34b44"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.863787 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 12:22:25.864634 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.864814 11790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:22:25.864912 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "28c15f86af824b1291ee7aafbff15ccd"
format_stamp: "Formatted at 2026-09-25 12:22:25 on dist-test-slave-l9ww"
server_key: "900c72687b9876281025809eb36be31b"
server_key_iv: "efd94d6fd2f0dcffeabbe36127d34b44"
server_key_version: "encryptionkey@0"
I20260925 12:22:25.864995 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:22:25.909778 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:22:25.912282 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:22:25.912446 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:22:25.912706 11790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:22:25.913017 11790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:22:25.913069 11790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.913146 11790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:22:25.913175 11790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:22:25.919044 11790 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:45111
I20260925 12:22:25.919126 11918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:45111 every 8 connection(s)
I20260925 12:22:25.919458 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:22:25 dist-test-slave-l9ww krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:22:25.928108 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 11790
I20260925 12:22:25.928205 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:22:25.928371 1214 external_mini_cluster.cc:1428] Setting key ba26584251b25c023a0faab49941c931
I20260925 12:22:25.929428 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.920368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:44811 (local address 127.1.47.190:41637)
0925 12:22:25.920429 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.920432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.920612 (+ 180us) server_negotiation.cc:407] Connection header received
0925 12:22:25.921160 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.921161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.921174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.921225 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:25.922171 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.922677 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.923844 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.924076 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.924165 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.926420 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:25.926421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:25.926422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:25.926429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:25.927903 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.928418 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.928420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.928420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.928485 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.928754 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.928755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.928755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.928892 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:25.928959 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.929256 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.929339 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 12:22:25.929641 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.920326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41637 (local address 127.1.47.131:44811)
0925 12:22:25.920488 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.920525 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.921038 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.921300 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.921304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.921862 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:22:25.922054 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.922059 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.922796 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.922797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.923753 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.923755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.924176 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.924177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.924301 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.924852 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:22:25.924861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:22:25.926325 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:22:25.928044 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.928050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.928052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.928343 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.928575 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:22:25.928576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:22:25.928577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:22:25.928707 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:22:25.928962 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:22:25.928968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:22:25.929114 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.929228 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.929382 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260925 12:22:25.930092 11919 heartbeater.cc:347] Connected to a master server at 127.1.47.190:41637
I20260925 12:22:25.930191 11919 heartbeater.cc:464] Registering TS with master...
I20260925 12:22:25.930357 11919 heartbeater.cc:511] Master 127.1.47.190:41637 requested a full tablet report, sending...
I20260925 12:22:25.930727 11460 ts_manager.cc:194] Registered new tserver with Master: 28c15f86af824b1291ee7aafbff15ccd (127.1.47.131:45111)
I20260925 12:22:25.931094 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:44811
I20260925 12:22:25.940310 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:22:25.948107 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.941632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.1.47.190:41637)
0925 12:22:25.941677 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.941679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.941828 (+ 149us) server_negotiation.cc:407] Connection header received
0925 12:22:25.941906 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.941907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.941918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.941974 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:22:25.942524 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.943030 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.943771 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.944032 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.944110 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.945414 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:22:25.945416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:22:25.945416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:22:25.945423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:22:25.946880 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.947254 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.947254 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.947255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.947314 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:22:25.947513 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:22:25.947514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:22:25.947514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:22:25.947675 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:22:25.947784 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.947924 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.948006 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:22:25.950004 11460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:22:25.951581 11460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:22:25.958196 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.953901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.1.47.130:44937)
0925 12:22:25.954144 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.954149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.954244 (+ 95us) server_negotiation.cc:407] Connection header received
0925 12:22:25.954385 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.954388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.954402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.954456 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:25.954679 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.955884 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957035 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.957781 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957861 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.957924 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.958061 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.958101 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":80,"threads_started":1}
I20260925 12:22:25.958180 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.953810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45111 (local address 127.0.0.1:52802)
0925 12:22:25.954200 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.954211 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.954279 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.954600 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.954601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.954603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:25.954677 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.954679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.955771 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.955772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.957234 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.957235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.957966 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.958024 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.958027 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.958082 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.958112 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":40,"thread_start_us":87,"threads_started":1}
I20260925 12:22:25.958374 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.954107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.1.47.129:38393)
0925 12:22:25.954292 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.954296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.954307 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:22:25.954352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.954354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.954369 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.954452 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:25.954721 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.955879 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957193 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.957873 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957931 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.957975 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.958247 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.958298 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260925 12:22:25.958621 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.953912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.1.47.131:45111)
0925 12:22:25.954161 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:25.954166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:25.954366 (+ 200us) server_negotiation.cc:407] Connection header received
0925 12:22:25.954408 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:25.954410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:25.954425 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:25.954489 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:25.954804 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.955666 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957320 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.957796 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957882 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.957936 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:25.958517 (+ 581us) server_negotiation.cc:299] Negotiation successful
0925 12:22:25.958564 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260925 12:22:25.958181 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.953805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:44937 (local address 127.0.0.1:60672)
0925 12:22:25.954128 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.954171 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.954275 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.954558 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.954559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.954561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:25.954599 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.954600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.956002 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.956003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.956955 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.956957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.957874 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.957875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.957949 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.957963 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.958037 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.958092 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260925 12:22:25.958348 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:25.954014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38393 (local address 127.0.0.1:33594)
0925 12:22:25.954210 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:25.954219 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:25.954275 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:25.954472 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:25.954474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:25.954476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:25.954565 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.954567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.955989 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.955990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.957105 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:25.957107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:25.958031 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:25.958033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:25.958102 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:25.958107 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:22:25.958160 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 12:22:25.958202 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":29,"thread_start_us":61,"threads_started":1}
I20260925 12:22:25.959426 11853 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddf741afdca4708b2d1aa836373b218 (DEFAULT_TABLE table=test-table [id=d12601711a87422fb3b374974fc374b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:25.959434 11719 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddf741afdca4708b2d1aa836373b218 (DEFAULT_TABLE table=test-table [id=d12601711a87422fb3b374974fc374b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:25.959432 11585 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddf741afdca4708b2d1aa836373b218 (DEFAULT_TABLE table=test-table [id=d12601711a87422fb3b374974fc374b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:22:25.959772 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddf741afdca4708b2d1aa836373b218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:25.959771 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddf741afdca4708b2d1aa836373b218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:25.959771 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddf741afdca4708b2d1aa836373b218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:22:25.962271 11936 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Bootstrap starting.
I20260925 12:22:25.963009 11937 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Bootstrap starting.
I20260925 12:22:25.963071 11936 tablet_bootstrap.cc:654] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:25.963662 11936 log.cc:824] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:25.963776 11937 tablet_bootstrap.cc:654] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:25.963994 11938 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: Bootstrap starting.
I20260925 12:22:25.964341 11937 log.cc:824] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:25.964579 11938 tablet_bootstrap.cc:654] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: Neither blocks nor log segments found. Creating new log.
I20260925 12:22:25.965029 11937 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: No bootstrap required, opened a new log
I20260925 12:22:25.965080 11938 log.cc:824] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: Log is configured to *not* fsync() on all Append() calls
I20260925 12:22:25.965119 11937 ts_tablet_manager.cc:1397] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:22:25.965847 11938 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: No bootstrap required, opened a new log
I20260925 12:22:25.965920 11938 ts_tablet_manager.cc:1397] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:22:25.967027 11937 raft_consensus.cc:348] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.967219 11937 raft_consensus.cc:374] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:25.967253 11937 raft_consensus.cc:729] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b581742c4c34eaf8660e237fad6ad33, State: Initialized, Role: FOLLOWER
I20260925 12:22:25.967372 11937 consensus_queue.cc:260] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.967700 11937 ts_tablet_manager.cc:1428] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:22:25.967707 11785 heartbeater.cc:503] Master 127.1.47.190:41637 was elected leader, sending a full tablet report...
I20260925 12:22:25.968000 11938 raft_consensus.cc:348] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.968151 11938 raft_consensus.cc:374] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:25.968185 11938 raft_consensus.cc:729] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad7b733d04a4bb2be8ef00778796914, State: Initialized, Role: FOLLOWER
I20260925 12:22:25.968241 11936 tablet_bootstrap.cc:492] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: No bootstrap required, opened a new log
I20260925 12:22:25.968297 11936 ts_tablet_manager.cc:1397] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260925 12:22:25.968294 11938 consensus_queue.cc:260] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.968559 11938 ts_tablet_manager.cc:1428] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:22:25.968684 11651 heartbeater.cc:503] Master 127.1.47.190:41637 was elected leader, sending a full tablet report...
I20260925 12:22:25.970132 11936 raft_consensus.cc:348] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.970278 11936 raft_consensus.cc:374] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:22:25.970323 11936 raft_consensus.cc:729] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c15f86af824b1291ee7aafbff15ccd, State: Initialized, Role: FOLLOWER
I20260925 12:22:25.970432 11936 consensus_queue.cc:260] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:25.970705 11936 ts_tablet_manager.cc:1428] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 12:22:25.970763 11919 heartbeater.cc:503] Master 127.1.47.190:41637 was elected leader, sending a full tablet report...
W20260925 12:22:25.979888 11786 tablet.cc:2369] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:22:26.028012 11943 raft_consensus.cc:482] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:22:26.028139 11943 raft_consensus.cc:504] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:26.028604 11943 leader_election.cc:288] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28c15f86af824b1291ee7aafbff15ccd (127.1.47.131:45111), 8b581742c4c34eaf8660e237fad6ad33 (127.1.47.130:44937)
I20260925 12:22:26.032523 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.028807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45111 (local address 127.1.47.129:39745)
0925 12:22:26.028876 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:26.028889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:26.028937 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:26.029170 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:26.029171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:26.029173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:26.029256 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:26.029258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.030169 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.030170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:26.031548 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:26.031551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.032252 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.032253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:26.032335 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:26.032338 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:22:26.032401 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 12:22:26.032449 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":102}
I20260925 12:22:26.032521 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.028913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:39745 (local address 127.1.47.131:45111)
0925 12:22:26.028961 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:26.029009 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:26.029027 (+ 18us) server_negotiation.cc:407] Connection header received
0925 12:22:26.029052 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:26.029053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:26.029081 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:26.029152 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:26.029334 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.030084 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.031671 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.032166 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.032253 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:26.032263 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:26.032419 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 12:22:26.032457 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:22:26.032704 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.028902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:56905 (local address 127.1.47.130:44937)
0925 12:22:26.028945 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:26.028947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:26.029098 (+ 151us) server_negotiation.cc:407] Connection header received
0925 12:22:26.029126 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:26.029127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:26.029139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:26.029200 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:22:26.029334 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.030080 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.031680 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.032209 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.032330 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:26.032341 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:26.032552 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 12:22:26.032596 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:22:26.032856 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddf741afdca4708b2d1aa836373b218" candidate_uuid: "fad7b733d04a4bb2be8ef00778796914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c15f86af824b1291ee7aafbff15ccd" is_pre_election: true
I20260925 12:22:26.032908 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.028823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:44937 (local address 127.1.47.129:56905)
0925 12:22:26.029015 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:26.029032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 12:22:26.029089 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:22:26.029202 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:22:26.029202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:22:26.029203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:22:26.029257 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:26.029258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.030169 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.030170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:26.031548 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:22:26.031551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.032330 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.032332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:22:26.032427 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:26.032433 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 12:22:26.032519 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 12:22:26.032570 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":64,"thread_start_us":117,"threads_started":1}
I20260925 12:22:26.033054 11873 raft_consensus.cc:2454] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fad7b733d04a4bb2be8ef00778796914 in term 0.
I20260925 12:22:26.033318 11539 leader_election.cc:302] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c15f86af824b1291ee7aafbff15ccd, fad7b733d04a4bb2be8ef00778796914; no voters:
I20260925 12:22:26.033336 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddf741afdca4708b2d1aa836373b218" candidate_uuid: "fad7b733d04a4bb2be8ef00778796914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b581742c4c34eaf8660e237fad6ad33" is_pre_election: true
I20260925 12:22:26.033486 11739 raft_consensus.cc:2454] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fad7b733d04a4bb2be8ef00778796914 in term 0.
I20260925 12:22:26.033490 11943 raft_consensus.cc:2781] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:22:26.033555 11943 raft_consensus.cc:482] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:22:26.033613 11943 raft_consensus.cc:3037] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:26.034329 11943 raft_consensus.cc:504] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:26.034631 11943 leader_election.cc:288] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [CANDIDATE]: Term 1 election: Requested vote from peers 28c15f86af824b1291ee7aafbff15ccd (127.1.47.131:45111), 8b581742c4c34eaf8660e237fad6ad33 (127.1.47.130:44937)
I20260925 12:22:26.034663 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddf741afdca4708b2d1aa836373b218" candidate_uuid: "fad7b733d04a4bb2be8ef00778796914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c15f86af824b1291ee7aafbff15ccd"
I20260925 12:22:26.034693 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddf741afdca4708b2d1aa836373b218" candidate_uuid: "fad7b733d04a4bb2be8ef00778796914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b581742c4c34eaf8660e237fad6ad33"
I20260925 12:22:26.034732 11873 raft_consensus.cc:3037] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:26.034799 11739 raft_consensus.cc:3037] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:22:26.035480 11873 raft_consensus.cc:2454] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fad7b733d04a4bb2be8ef00778796914 in term 1.
I20260925 12:22:26.035655 11539 leader_election.cc:302] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c15f86af824b1291ee7aafbff15ccd, fad7b733d04a4bb2be8ef00778796914; no voters:
I20260925 12:22:26.035683 11739 raft_consensus.cc:2454] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fad7b733d04a4bb2be8ef00778796914 in term 1.
I20260925 12:22:26.035912 11943 raft_consensus.cc:2781] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:22:26.036121 11943 raft_consensus.cc:686] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [term 1 LEADER]: Becoming Leader. State: Replica: fad7b733d04a4bb2be8ef00778796914, State: Running, Role: LEADER
I20260925 12:22:26.036230 11943 consensus_queue.cc:237] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } }
I20260925 12:22:26.037199 11459 catalog_manager.cc:5495] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 reported cstate change: term changed from 0 to 1, leader changed from <none> to fad7b733d04a4bb2be8ef00778796914 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "fad7b733d04a4bb2be8ef00778796914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fad7b733d04a4bb2be8ef00778796914" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38393 } health_report { overall_health: HEALTHY } } }
I20260925 12:22:26.054798 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.052478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33604 (local address 127.1.47.129:38393)
0925 12:22:26.052543 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:22:26.052545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:22:26.052563 (+ 18us) server_negotiation.cc:407] Connection header received
0925 12:22:26.052597 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:22:26.052598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:22:26.052617 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:22:26.052684 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:22:26.052809 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.053276 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.054027 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:22:26.054230 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:22:26.054294 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:22:26.054422 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:22:26.054609 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:22:26.054686 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:22:26.054705 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 12:22:26.054724 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:22:26.056663 11873 raft_consensus.cc:1261] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 1 FOLLOWER]: Refusing update from remote peer fad7b733d04a4bb2be8ef00778796914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:26.056663 11739 raft_consensus.cc:1261] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 1 FOLLOWER]: Refusing update from remote peer fad7b733d04a4bb2be8ef00778796914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:22:26.057011 11943 consensus_queue.cc:1035] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b581742c4c34eaf8660e237fad6ad33" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:26.057137 11946 consensus_queue.cc:1035] T 5ddf741afdca4708b2d1aa836373b218 P fad7b733d04a4bb2be8ef00778796914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c15f86af824b1291ee7aafbff15ccd" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:22:26.060719 11951 mvcc.cc:205] Tried to move back new op lower bound from 7333228323045470208 to 7333228322965032960. Current Snapshot: MvccSnapshot[applied={T|T < 7333228323045470208 or (T in {7333228323045470208})}]
I20260925 12:22:26.060837 11950 mvcc.cc:205] Tried to move back new op lower bound from 7333228323045470208 to 7333228322965032960. Current Snapshot: MvccSnapshot[applied={T|T < 7333228323045470208 or (T in {7333228323045470208})}]
I20260925 12:22:26.061753 11460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:26.061847 11460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:22:26.062512 11460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc2cd144c4994399a3d4ae13a7a9ee25: Sending DeleteTablet for 3 replicas of tablet 5ddf741afdca4708b2d1aa836373b218
I20260925 12:22:26.062840 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddf741afdca4708b2d1aa836373b218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:26 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:60672
I20260925 12:22:26.062849 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddf741afdca4708b2d1aa836373b218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:26 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:52802
I20260925 12:22:26.063040 11958 tablet_replica.cc:330] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: stopping tablet replica
I20260925 12:22:26.063169 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11522
I20260925 12:22:26.063170 11958 raft_consensus.cc:2229] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:26.063218 11585 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddf741afdca4708b2d1aa836373b218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:22:26 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:33594
I20260925 12:22:26.065470 11958 raft_consensus.cc:2258] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:26.065896 11958 ts_tablet_manager.cc:1905] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:26.067449 11958 ts_tablet_manager.cc:1918] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:26.067528 11958 log.cc:1196] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/5ddf741afdca4708b2d1aa836373b218
W20260925 12:22:26.067878 11445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:38393 (error 108)
I20260925 12:22:26.068003 11958 ts_tablet_manager.cc:1939] T 5ddf741afdca4708b2d1aa836373b218 P 8b581742c4c34eaf8660e237fad6ad33: Deleting consensus metadata
I20260925 12:22:26.068092 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11656
I20260925 12:22:26.068125 11959 tablet_replica.cc:330] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: stopping tablet replica
I20260925 12:22:26.068221 11959 raft_consensus.cc:2229] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:22:26.068313 11959 raft_consensus.cc:2258] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:22:26.069904 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.069757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38393 (local address 127.0.0.1:33610)
0925 12:22:26.069822 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:26.069865 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 12:22:26.070007 11445 catalog_manager.cc:4584] TS fad7b733d04a4bb2be8ef00778796914 (127.1.47.129:38393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ddf741afdca4708b2d1aa836373b218: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38393: connect: Connection refused (error 111)
I20260925 12:22:26.070117 11959 ts_tablet_manager.cc:1905] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:22:26.072062 11959 ts_tablet_manager.cc:1918] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:22:26.072140 11959 log.cc:1196] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/5ddf741afdca4708b2d1aa836373b218
I20260925 12:22:26.072420 11959 ts_tablet_manager.cc:1939] T 5ddf741afdca4708b2d1aa836373b218 P 28c15f86af824b1291ee7aafbff15ccd: Deleting consensus metadata
I20260925 12:22:26.072889 11445 catalog_manager.cc:4841] TS 28c15f86af824b1291ee7aafbff15ccd (127.1.47.131:45111): tablet 5ddf741afdca4708b2d1aa836373b218 (table test-table [id=d12601711a87422fb3b374974fc374b5]) successfully deleted
I20260925 12:22:26.073822 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11790
I20260925 12:22:26.073880 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:22:26.073750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:44937 (local address 127.0.0.1:60684)
0925 12:22:26.073821 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 12:22:26.073847 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.130:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 12:22:26.078686 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 11428
2026-09-25T12:22:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1966 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:22:26 dist-test-slave-l9ww krb5kdc[11963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:22:26 dist-test-slave-l9ww krb5kdc[11963](info): set up 2 sockets
Sep 25 12:22:26 dist-test-slave-l9ww krb5kdc[11963](info): commencing operation
krb5kdc: starting...
W20260925 12:22:27.125490 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:22:27 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:22:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:22:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangeradmin_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangeradmin_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangerlookup_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangerlookup_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
I20260925 12:22:27.290222 1214 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuGajnc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/postgres -l logfile start
2026-09-25 12:22:29.889 UTC [11993] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 12:22:29.889 UTC [11993] LOG: listening on IPv4 address "127.1.47.148", port 45369
2026-09-25 12:22:29.893 UTC [11993] LOG: listening on Unix socket "/tmp/.s.PGSQL.45369"
2026-09-25 12:22:29.916 UTC [11996] LOG: database system was shut down at 2026-09-25 12:22:28 UTC
2026-09-25 12:22:29.920 UTC [11993] LOG: database system is ready to accept connections
I20260925 12:22:30.877010 1214 mini_postgres.cc:96] Postgres bound to 45369
2026-09-25 12:22:30.883 UTC [12004] FATAL: database "slave" does not exist
127.1.47.148:45369 - accepting connections
I20260925 12:22:30.884225 1214 mini_ranger.cc:161] Starting Ranger...
I20260925 12:22:30.900601 1214 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 12:22:31.420239 1214 mini_ranger.cc:87] Created ranger Postgres database
I20260925 12:22:31.420308 1214 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin
2026-09-25 12:22:31,706 [I] DB FLAVOR :POSTGRES
2026-09-25 12:22:31,707 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 12:22:31,707 [I] Checking connection..
2026-09-25 12:22:31,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 12:22:31,960 [I] Checking connection passed.
2026-09-25 12:22:31,961 [I] --------- Verifying version history table ---------
2026-09-25 12:22:31,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:22:32,201 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 12:22:32,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:22:32,471 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 12:22:32,471 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 12:22:32,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 12:22:32.749 UTC [12068] WARNING: there is no transaction in progress
2026-09-25 12:22:32,756 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 12:22:32,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:22:33,024 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 12:22:33,024 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 12:22:33,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 12:22:33,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 12:22:33,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l9ww.c.gcp-upstream.internal','N') ;"
2026-09-25 12:22:33,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 12:22:34,011 [I] Table x_portal_user does not exist in database ranger
2026-09-25 12:22:34,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 12:22:34,250 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 12:22:34,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 12:22:34,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 12:22:34,739 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 12:22:34,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 12:22:35.801 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.819 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.838 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.854 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.874 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.896 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.959 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.967 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.984 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:35.995 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.006 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.021 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.032 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.040 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.048 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36.055 UTC [12176] WARNING: there is no transaction in progress
2026-09-25 12:22:36,754 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 12:22:36,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 12:22:37,006 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 12:22:37,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 12:22:37,244 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 12:22:37,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 12:22:37,525 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 12:22:37,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 12:22:37,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 12:22:37,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l9ww.c.gcp-upstream.internal';"
2026-09-25 12:22:38,251 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 12:22:38,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 12:22:38,517 [I] DB_PATCHES have already been applied
I20260925 12:22:38.522853 1214 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuGajnc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 12:22:38.522919 1214 mini_ranger.cc:193] Using host: 127.1.47.148
I20260925 12:22:38.525971 1214 mini_ranger.cc:227] Ranger admin URL: http://127.1.47.148:44135
Sep 25, 2026 12:22:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:22:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:22:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 12:22:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 12:22:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 12:22:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 12:22:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 12:22:39.643636 1214 mini_ranger.cc:160] Time spent starting Ranger: real 8.759s user 0.004s sys 0.005s
Sep 25, 2026 12:23:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 12:23:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 12:23:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 12:23:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 12:23:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 12:23:02.481046 1214 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangerkms_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/rangerkms_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 12:23:02 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 12:23:02.565858 1214 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 12:23:02.575320 1214 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 12:23:03.437685 1214 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 12:23:03.437781 1214 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms
I20260925 12:23:03.437804 1214 mini_ranger_kms.cc:227] Using postgres at 127.1.47.148:45369
2026-09-25 12:23:03,619 [I] DB FLAVOR :POSTGRES
2026-09-25 12:23:03,620 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 12:23:03,620 [I] Checking connection
2026-09-25 12:23:03,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 12:23:03,866 [I] connection success
2026-09-25 12:23:03,866 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 12:23:03,866 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 12:23:03,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 12:23:04,105 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 12:23:04,105 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 12:23:04,105 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 12:23:04,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:45369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 12:23:04,385 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 12:23:04.872583 1214 mini_ranger_kms.cc:310] Created kms service
I20260925 12:23:05.110340 1214 mini_ranger_kms.cc:326] Created kudu user
I20260925 12:23:05.155162 1214 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 12:23:05 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:05.578575 1214 mini_ranger_kms.cc:384] Added ranger policy
I20260925 12:23:05.578711 1214 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 12:23:05.578742 1214 mini_ranger_kms.cc:240] Using host: 127.1.47.148
I20260925 12:23:05.581955 1214 mini_ranger_kms.cc:276] Ranger KMS PID: 12357
I20260925 12:23:05.582094 1214 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.47.148:47517
Sep 25, 2026 12:23:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:23:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:23:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 12:23:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 12:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 12:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 12:23:07.868548 1214 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.303s user 0.000s sys 0.164s
I20260925 12:23:07.868677 1214 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 12:23:07.868774 1214 mini_ranger_kms.cc:401] 127.1.47.148:47517/kms/v1/keys
Sep 25 12:23:07 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790338987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 12:23:08 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
Sep 25 12:23:08 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
Sep 25 12:23:10 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:23:11.035037 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43383
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:43383
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:47517/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:23:11.132730 12407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:23:11.132956 12407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:23:11.132977 12407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:23:11.134286 12407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:23:11.134335 12407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:23:11.134349 12407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:23:11.134385 12407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:23:11.134418 12407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:23:11.136457 12407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.47.148:47517/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:43383
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:43383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:23:11.136734 12407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:23:11.136978 12407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:23:11.142385 12412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:11.142385 12415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:11.142442 12407 server_base.cc:1034] running on GCE node
W20260925 12:23:11.142436 12413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:11.142931 12407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:23:11.143227 12407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:23:11.144380 12407 hybrid_clock.cc:648] HybridClock initialized: now 1790338991144366 us; error 28 us; skew 500 ppm
Sep 25 12:23:11 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:11.146903 12407 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:23:11.147400 12407 webserver.cc:460] Webserver started at http://127.1.47.190:39509/ using document root <none> and password file <none>
I20260925 12:23:11.147619 12407 fs_manager.cc:359] Metadata directory not provided
I20260925 12:23:11.147676 12407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:23:11.147819 12407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:23:11 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:11.205479 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "9784e1adfaf34b7690da4bc0eec5828e"
format_stamp: "Formatted at 2026-09-25 12:23:11 on dist-test-slave-l9ww"
server_key: "69c34565d1ab72ecafe231f4d2fb6b23"
server_key_iv: "42ff85bd3181830e136c650137cd1ad6"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:11.206008 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "9784e1adfaf34b7690da4bc0eec5828e"
format_stamp: "Formatted at 2026-09-25 12:23:11 on dist-test-slave-l9ww"
server_key: "69c34565d1ab72ecafe231f4d2fb6b23"
server_key_iv: "42ff85bd3181830e136c650137cd1ad6"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:11.207975 12407 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 12:23:11.224084 12422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:11.224344 12407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 12:23:11.224465 12407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "9784e1adfaf34b7690da4bc0eec5828e"
format_stamp: "Formatted at 2026-09-25 12:23:11 on dist-test-slave-l9ww"
server_key: "69c34565d1ab72ecafe231f4d2fb6b23"
server_key_iv: "42ff85bd3181830e136c650137cd1ad6"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:11.224571 12407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:23:11.238332 12407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:23:11.240581 12407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:23:11.240733 12407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:23:11.246261 12407 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:43383
I20260925 12:23:11.246331 12474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:43383 every 8 connection(s)
I20260925 12:23:11.246731 12407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:23:11.247453 12475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:23:11.249801 12475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: Bootstrap starting.
I20260925 12:23:11.250396 12475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: Neither blocks nor log segments found. Creating new log.
I20260925 12:23:11.250680 12475 log.cc:824] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:23:11.251379 12475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: No bootstrap required, opened a new log
I20260925 12:23:11.251757 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 12407
I20260925 12:23:11.251888 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:23:11.252075 1214 external_mini_cluster.cc:1428] Setting key 43e96f4ffb8158c685c81bdef8d14109
I20260925 12:23:11.254302 12475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } }
I20260925 12:23:11.254458 12475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:23:11.254519 12475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9784e1adfaf34b7690da4bc0eec5828e, State: Initialized, Role: FOLLOWER
I20260925 12:23:11.254671 12475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } }
I20260925 12:23:11.254767 12475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:23:11.254823 12475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:23:11.254895 12475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:23:11.255451 12475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } }
I20260925 12:23:11.255574 12475 leader_election.cc:302] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9784e1adfaf34b7690da4bc0eec5828e; no voters:
I20260925 12:23:11.255743 12475 leader_election.cc:288] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:23:11.255785 12480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:23:11.255997 12480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [term 1 LEADER]: Becoming Leader. State: Replica: 9784e1adfaf34b7690da4bc0eec5828e, State: Running, Role: LEADER
I20260925 12:23:11.256093 12475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:23:11.256145 12480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } }
I20260925 12:23:11.256508 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9784e1adfaf34b7690da4bc0eec5828e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } } }
I20260925 12:23:11.256672 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [sys.catalog]: This master's current role is: LEADER
I20260925 12:23:11.256879 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9784e1adfaf34b7690da4bc0eec5828e. Latest consensus state: current_term: 1 leader_uuid: "9784e1adfaf34b7690da4bc0eec5828e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9784e1adfaf34b7690da4bc0eec5828e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 43383 } } }
I20260925 12:23:11.256992 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e [sys.catalog]: This master's current role is: LEADER
I20260925 12:23:11.257149 12489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:23:11.258153 12489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:23:11.258934 12475 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 25 12:23:11 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:23:11.267989 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:11.252605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.1.47.190:43383)
0925 12:23:11.252815 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:11.252825 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:11.252856 (+ 31us) server_negotiation.cc:407] Connection header received
0925 12:23:11.253829 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:11.253839 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:11.253905 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:11.254067 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:11.254859 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:11.256343 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:11.257445 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:11.258469 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:11.258580 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:11.262816 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:11.262882 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:11.262883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:11.262992 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:11.265603 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:11.266008 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:11.266011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:11.266012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:11.266148 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:11.266367 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:11.266368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:11.266368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:11.267045 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:11.267216 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:11.267330 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 12:23:11.267488 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260925 12:23:11.272411 12489 catalog_manager.cc:1347] Generated new cluster ID: 53926947608c4ab2820c107eca68de9b
I20260925 12:23:11.272478 12489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:23:11.291744 12489 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:23:11.292610 12489 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:23:11.296490 12489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: Generated new TSK 0
I20260925 12:23:11.296753 12489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:23:12.038416 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:47517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:23:12.130734 12520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:23:12.130967 12520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:23:12.131011 12520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:23:12.132308 12520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:23:12.132371 12520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:23:12.132416 12520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:23:12.133934 12520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.47.148:47517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:23:12.134215 12520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:23:12.134470 12520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:23:12.139941 12526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.139941 12528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.140403 12525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:12.141091 12520 server_base.cc:1034] running on GCE node
I20260925 12:23:12.141346 12520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:23:12.141605 12520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:23:12.144860 12520 hybrid_clock.cc:648] HybridClock initialized: now 1790338992144846 us; error 31 us; skew 500 ppm
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:12.147966 12520 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:23:12.148489 12520 webserver.cc:460] Webserver started at http://127.1.47.129:39867/ using document root <none> and password file <none>
I20260925 12:23:12.148761 12520 fs_manager.cc:359] Metadata directory not provided
I20260925 12:23:12.148831 12520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:23:12.148976 12520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:12.164139 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "2a17551c468e4abda8e7cbeb840ab68b"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "6657de4f7301bd28d828313e99a25ac0"
server_key_iv: "f5423911bbe1b2e03b12821f75a25e78"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.164625 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "2a17551c468e4abda8e7cbeb840ab68b"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "6657de4f7301bd28d828313e99a25ac0"
server_key_iv: "f5423911bbe1b2e03b12821f75a25e78"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.168289 12520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 12:23:12.185813 12535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.186113 12520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 12:23:12.186264 12520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "2a17551c468e4abda8e7cbeb840ab68b"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "6657de4f7301bd28d828313e99a25ac0"
server_key_iv: "f5423911bbe1b2e03b12821f75a25e78"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.186420 12520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:23:12.224215 12520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:23:12.226818 12520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:23:12.227015 12520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:23:12.227350 12520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:23:12.227789 12520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:23:12.227897 12520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.227999 12520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:23:12.228071 12520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.235466 12520 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38555
I20260925 12:23:12.236007 12520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:23:12.237264 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 12520
I20260925 12:23:12.237373 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:23:12.237560 1214 external_mini_cluster.cc:1428] Setting key 4c7df465592b9702f2021b14b38870ea
I20260925 12:23:12.241990 12650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38555 every 8 connection(s)
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
I20260925 12:23:12.259064 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.242912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43383 (local address 127.1.47.129:43285)
0925 12:23:12.243498 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.243547 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.244209 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.244647 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.244651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.244902 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:23:12.245205 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.245211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.246277 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.246278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.247569 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.247572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.248139 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.248140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.248303 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.248989 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:23:12.249008 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:23:12.254667 (+ 5659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:23:12.257174 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.257196 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.257200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.257559 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.257833 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.257834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.257835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.257997 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.258319 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:23:12.258325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:23:12.258528 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.258646 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.258816 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":95,"threads_started":1}
I20260925 12:23:12.259529 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.243222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:43285 (local address 127.1.47.190:43383)
0925 12:23:12.244403 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.244406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.244414 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:23:12.244441 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.244442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.244454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.244516 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:12.245340 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.246141 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.247674 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.247995 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.248065 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.254847 (+ 6782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:12.254849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:12.254849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:12.254858 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:12.257047 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.257643 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.257645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.257645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.257765 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.258062 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.258063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.258063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.258234 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:12.258293 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.259353 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.259440 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1151,"thread_start_us":66,"threads_started":1}
I20260925 12:23:12.259912 12651 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43383
I20260925 12:23:12.260020 12651 heartbeater.cc:464] Registering TS with master...
I20260925 12:23:12.260243 12651 heartbeater.cc:511] Master 127.1.47.190:43383 requested a full tablet report, sending...
I20260925 12:23:12.260794 12439 ts_manager.cc:194] Registered new tserver with Master: 2a17551c468e4abda8e7cbeb840ab68b (127.1.47.129:38555)
I20260925 12:23:12.261716 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:43285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:23:12.284664 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:47517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:23:12.406132 12657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:23:12.406391 12657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:23:12.406435 12657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:23:12.407787 12657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:23:12.407857 12657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:23:12.407958 12657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:23:12.409799 12657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.47.148:47517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:23:12.410072 12657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:23:12.410331 12657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:23:12.415550 12667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.415608 12665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.415925 12664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:12.416173 12657 server_base.cc:1034] running on GCE node
I20260925 12:23:12.416410 12657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:23:12.416672 12657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:12.419950 12657 hybrid_clock.cc:648] HybridClock initialized: now 1790338992419906 us; error 31 us; skew 500 ppm
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:12.424201 12657 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:23:12.424813 12657 webserver.cc:460] Webserver started at http://127.1.47.130:46557/ using document root <none> and password file <none>
I20260925 12:23:12.425274 12657 fs_manager.cc:359] Metadata directory not provided
I20260925 12:23:12.425400 12657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:23:12.425606 12657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:12.448180 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "02b3db1c65f942f29152ce3008ba2c89"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "47cc78167ebab44a91edbc17b049ebf4"
server_key_iv: "518f65f320d71bba2c34401836996c5d"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.448660 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "02b3db1c65f942f29152ce3008ba2c89"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "47cc78167ebab44a91edbc17b049ebf4"
server_key_iv: "518f65f320d71bba2c34401836996c5d"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.455497 12657 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260925 12:23:12.472070 12674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.472293 12657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 12:23:12.472409 12657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "02b3db1c65f942f29152ce3008ba2c89"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "47cc78167ebab44a91edbc17b049ebf4"
server_key_iv: "518f65f320d71bba2c34401836996c5d"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.472527 12657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:23:12.481993 12657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:23:12.484299 12657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:23:12.484475 12657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:23:12.484748 12657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:23:12.485118 12657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:23:12.485169 12657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.485210 12657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:23:12.485232 12657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.492123 12657 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:37287
I20260925 12:23:12.492610 12657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:23:12.493041 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 12657
I20260925 12:23:12.493178 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:23:12.493345 1214 external_mini_cluster.cc:1428] Setting key 6de6523c54909e60bbc7963d9a63c1de
I20260925 12:23:12.492821 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:37287 every 8 connection(s)
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
I20260925 12:23:12.505079 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.495584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:49681 (local address 127.1.47.190:43383)
0925 12:23:12.495646 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.495648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.495660 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:23:12.495805 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.495806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.495822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.495944 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:12.496772 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.497586 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.499102 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.499362 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.499446 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.502055 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:12.502057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:12.502057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:12.502065 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:12.503535 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.504026 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.504028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.504028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.504090 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.504351 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.504352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.504353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.504476 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:12.504551 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.504915 (+ 364us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.505008 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 12:23:12.505721 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.494660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43383 (local address 127.1.47.130:49681)
0925 12:23:12.494891 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.494947 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.495688 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.496051 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.496056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.496362 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:23:12.496652 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.496659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.497695 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.497697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.499014 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.499017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.499514 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.499515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.499631 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.500246 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:23:12.500519 (+ 273us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:23:12.501953 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:23:12.503639 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.503647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.503650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.503963 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.504155 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.504156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.504156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.504282 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.504541 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:23:12.504548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:23:12.504706 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.505246 (+ 540us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.505422 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
I20260925 12:23:12.506227 12788 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43383
I20260925 12:23:12.506330 12788 heartbeater.cc:464] Registering TS with master...
I20260925 12:23:12.506536 12788 heartbeater.cc:511] Master 127.1.47.190:43383 requested a full tablet report, sending...
I20260925 12:23:12.506930 12439 ts_manager.cc:194] Registered new tserver with Master: 02b3db1c65f942f29152ce3008ba2c89 (127.1.47.130:37287)
I20260925 12:23:12.507444 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:49681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:23:12.522683 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:47517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:23:12.622951 12794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:23:12.623342 12794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:23:12.623422 12794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:23:12.625267 12794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:23:12.625339 12794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:23:12.625380 12794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:23:12.627653 12794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.47.148:47517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:23:12.627957 12794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:23:12.628261 12794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:23:12.635130 12802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.635141 12800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:12.635593 12799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:12.635788 12794 server_base.cc:1034] running on GCE node
I20260925 12:23:12.636034 12794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:23:12.636348 12794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:23:12.637519 12794 hybrid_clock.cc:648] HybridClock initialized: now 1790338992637492 us; error 41 us; skew 500 ppm
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:12.640836 12794 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:23:12.641396 12794 webserver.cc:460] Webserver started at http://127.1.47.131:35263/ using document root <none> and password file <none>
I20260925 12:23:12.641674 12794 fs_manager.cc:359] Metadata directory not provided
I20260925 12:23:12.641755 12794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:23:12.641906 12794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:12.654872 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "3acbc50d382e4feca4bcd69a66a02371"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "ca7cbc091c0ca761090d7665592c886d"
server_key_iv: "296b45d15ed68a6e02ff4757ef33e371"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.655483 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "3acbc50d382e4feca4bcd69a66a02371"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "ca7cbc091c0ca761090d7665592c886d"
server_key_iv: "296b45d15ed68a6e02ff4757ef33e371"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.671221 12794 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.000s
I20260925 12:23:12.685390 12809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.686000 12794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:23:12.686100 12794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "3acbc50d382e4feca4bcd69a66a02371"
format_stamp: "Formatted at 2026-09-25 12:23:12 on dist-test-slave-l9ww"
server_key: "ca7cbc091c0ca761090d7665592c886d"
server_key_iv: "296b45d15ed68a6e02ff4757ef33e371"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:12.686213 12794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:23:12.701388 12794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:23:12.703537 12794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:23:12.703702 12794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:23:12.703974 12794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:23:12.704342 12794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:23:12.704392 12794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.704437 12794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:23:12.704463 12794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:12.711479 12794 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38663
I20260925 12:23:12.711777 12922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38663 every 8 connection(s)
I20260925 12:23:12.711927 12794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:23:12.718869 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 12794
I20260925 12:23:12.718981 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:23:12.719206 1214 external_mini_cluster.cc:1428] Setting key e056962336268d4b23275c4f7306a247
I20260925 12:23:12.722579 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.712708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:57117 (local address 127.1.47.190:43383)
0925 12:23:12.712782 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.712784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.713043 (+ 259us) server_negotiation.cc:407] Connection header received
0925 12:23:12.713579 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.713580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.713601 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.713653 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:12.714378 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.714888 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.715943 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.716147 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.716229 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.719149 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:12.719151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:12.719152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:12.719161 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:12.720963 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.721472 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.721473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.721473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.721534 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.721835 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.721836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.721836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.721975 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:12.722024 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.722404 (+ 380us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.722490 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 12:23:12.723091 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.712683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:43383 (local address 127.1.47.131:57117)
0925 12:23:12.712934 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.712965 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.713473 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.713742 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.713746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.714034 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:23:12.714243 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.714249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.714981 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.714982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.715807 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.715809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.716240 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.716241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.716367 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.716907 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:23:12.716916 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:23:12.718985 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:23:12.721060 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.721068 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.721071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.721377 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.721622 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:23:12.721622 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:23:12.721623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:23:12.721759 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:23:12.722074 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:23:12.722080 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:23:12.722250 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.722687 (+ 437us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.722858 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":93,"threads_started":1}
I20260925 12:23:12.723526 12923 heartbeater.cc:347] Connected to a master server at 127.1.47.190:43383
I20260925 12:23:12.723625 12923 heartbeater.cc:464] Registering TS with master...
I20260925 12:23:12.723812 12923 heartbeater.cc:511] Master 127.1.47.190:43383 requested a full tablet report, sending...
I20260925 12:23:12.724165 12439 ts_manager.cc:194] Registered new tserver with Master: 3acbc50d382e4feca4bcd69a66a02371 (127.1.47.131:38663)
I20260925 12:23:12.724687 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:57117
I20260925 12:23:12.738971 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:23:12.752616 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.740776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.1.47.190:43383)
0925 12:23:12.740848 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.740851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.740956 (+ 105us) server_negotiation.cc:407] Connection header received
0925 12:23:12.741016 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.741017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.741029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.741105 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:12.741826 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.742626 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.744756 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.745068 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.745140 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.746900 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:12.746902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:12.746902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:12.746912 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:12.748748 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.751520 (+ 2772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.751522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.751522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.751603 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:12.751877 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:12.751878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:12.751878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:12.752034 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:12.752117 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.752415 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.752512 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:23:12.755043 12439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:23:12.757462 12439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:23:12.766309 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:37287 (local address 127.0.0.1:51442)
0925 12:23:12.760631 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.760673 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.760740 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.761444 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.761447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.761450 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:23:12.761523 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.761525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.762870 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.762871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.764522 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.764525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.765984 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.765985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.766073 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.766089 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.766165 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.766221 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":80,"threads_started":1}
I20260925 12:23:12.766674 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.1.47.130:37287)
0925 12:23:12.760890 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.760894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.760905 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:23:12.760944 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.760946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.760962 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.761347 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:23:12.761618 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.762770 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.764656 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.765793 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.765866 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.765947 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.766531 (+ 584us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.766573 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":79,"threads_started":1}
I20260925 12:23:12.767379 12722 tablet_service.cc:1467] Processing CreateTablet for tablet b348148259c64a7f934ec2d2097c8ef7 (DEFAULT_TABLE table=test-table [id=41c4f09af1a748c8864469f85ac8b350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:23:12.767799 12722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b348148259c64a7f934ec2d2097c8ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:23:12.769088 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38663 (local address 127.0.0.1:57942)
0925 12:23:12.761055 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.761066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.761118 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.761571 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.761572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.761574 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:23:12.761620 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.761621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.764000 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.764001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.765077 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.765079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.768833 (+ 3754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.768834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.768904 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.768908 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.768967 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.769010 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":79,"threads_started":1}
I20260925 12:23:12.770476 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38555 (local address 127.0.0.1:56936)
0925 12:23:12.761210 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.761218 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.761253 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.761949 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.761950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.761952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:23:12.761986 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.761987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.765788 (+ 3801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.765789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.767283 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.767286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.770222 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.770223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.770289 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.770294 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.770349 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.770396 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":61,"threads_started":1}
I20260925 12:23:12.770931 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.1.47.129:38555)
0925 12:23:12.761730 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.761734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.761743 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:23:12.761786 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.761788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.761803 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.761872 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:23:12.762060 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.765690 (+ 3630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.767663 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.770033 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.770107 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.770170 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.770809 (+ 639us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.770858 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":46,"threads_started":1}
I20260925 12:23:12.771291 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.760732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.1.47.131:38663)
0925 12:23:12.761360 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.761365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.761374 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:23:12.761411 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.761413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.761428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.761497 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:23:12.762816 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.763893 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.767408 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.768634 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.768711 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.768789 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.771136 (+ 2347us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.771202 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":76,"threads_started":1}
I20260925 12:23:12.771982 12941 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Bootstrap starting.
I20260925 12:23:12.772100 12574 tablet_service.cc:1467] Processing CreateTablet for tablet b348148259c64a7f934ec2d2097c8ef7 (DEFAULT_TABLE table=test-table [id=41c4f09af1a748c8864469f85ac8b350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:23:12.772469 12574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b348148259c64a7f934ec2d2097c8ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:23:12.772985 12941 tablet_bootstrap.cc:654] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Neither blocks nor log segments found. Creating new log.
I20260925 12:23:12.773267 12941 log.cc:824] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Log is configured to *not* fsync() on all Append() calls
I20260925 12:23:12.774012 12941 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: No bootstrap required, opened a new log
I20260925 12:23:12.774070 12941 ts_tablet_manager.cc:1397] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:23:12.776453 12941 raft_consensus.cc:348] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.776638 12941 raft_consensus.cc:374] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:23:12.776677 12941 raft_consensus.cc:729] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02b3db1c65f942f29152ce3008ba2c89, State: Initialized, Role: FOLLOWER
I20260925 12:23:12.776945 12857 tablet_service.cc:1467] Processing CreateTablet for tablet b348148259c64a7f934ec2d2097c8ef7 (DEFAULT_TABLE table=test-table [id=41c4f09af1a748c8864469f85ac8b350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:23:12.776783 12941 consensus_queue.cc:260] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.777295 12857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b348148259c64a7f934ec2d2097c8ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:23:12.777529 12941 ts_tablet_manager.cc:1428] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:23:12.777659 12788 heartbeater.cc:503] Master 127.1.47.190:43383 was elected leader, sending a full tablet report...
I20260925 12:23:12.780555 12945 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Bootstrap starting.
I20260925 12:23:12.781097 12943 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: Bootstrap starting.
I20260925 12:23:12.781862 12943 tablet_bootstrap.cc:654] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: Neither blocks nor log segments found. Creating new log.
I20260925 12:23:12.781936 12945 tablet_bootstrap.cc:654] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Neither blocks nor log segments found. Creating new log.
I20260925 12:23:12.782236 12945 log.cc:824] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Log is configured to *not* fsync() on all Append() calls
I20260925 12:23:12.782333 12943 log.cc:824] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: Log is configured to *not* fsync() on all Append() calls
I20260925 12:23:12.782972 12945 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: No bootstrap required, opened a new log
I20260925 12:23:12.783037 12945 ts_tablet_manager.cc:1397] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:23:12.783100 12943 tablet_bootstrap.cc:492] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: No bootstrap required, opened a new log
I20260925 12:23:12.783171 12943 ts_tablet_manager.cc:1397] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:23:12.785405 12945 raft_consensus.cc:348] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.785564 12945 raft_consensus.cc:374] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:23:12.785606 12945 raft_consensus.cc:729] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbc50d382e4feca4bcd69a66a02371, State: Initialized, Role: FOLLOWER
I20260925 12:23:12.785746 12945 consensus_queue.cc:260] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.785981 12945 ts_tablet_manager.cc:1428] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:23:12.786082 12923 heartbeater.cc:503] Master 127.1.47.190:43383 was elected leader, sending a full tablet report...
I20260925 12:23:12.785979 12943 raft_consensus.cc:348] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.786141 12943 raft_consensus.cc:374] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:23:12.786175 12943 raft_consensus.cc:729] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a17551c468e4abda8e7cbeb840ab68b, State: Initialized, Role: FOLLOWER
I20260925 12:23:12.786475 12943 consensus_queue.cc:260] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.786890 12943 ts_tablet_manager.cc:1428] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 12:23:12.787055 12651 heartbeater.cc:503] Master 127.1.47.190:43383 was elected leader, sending a full tablet report...
Sep 25 12:23:12 dist-test-slave-l9ww krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338992, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:12.885862 12949 raft_consensus.cc:482] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:23:12.886029 12949 raft_consensus.cc:504] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.886498 12949 leader_election.cc:288] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02b3db1c65f942f29152ce3008ba2c89 (127.1.47.130:37287), 3acbc50d382e4feca4bcd69a66a02371 (127.1.47.131:38663)
I20260925 12:23:12.895516 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.886826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:49695 (local address 127.1.47.130:37287)
0925 12:23:12.886936 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.886938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.887946 (+ 1008us) server_negotiation.cc:407] Connection header received
0925 12:23:12.889526 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.889528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.889544 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.889576 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:23:12.890489 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.891274 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.894365 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.894894 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.894942 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.894951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.895389 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.895437 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 12:23:12.895965 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.886676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:37287 (local address 127.1.47.129:49695)
0925 12:23:12.887837 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.887851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.887981 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.889975 (+ 1994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.889976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.889991 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:23:12.890076 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.890079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.891656 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.891657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.893999 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.894273 (+ 274us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.895218 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.895219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.895308 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.895312 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.895813 (+ 501us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.895883 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":121,"threads_started":1}
I20260925 12:23:12.896466 12742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b348148259c64a7f934ec2d2097c8ef7" candidate_uuid: "2a17551c468e4abda8e7cbeb840ab68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02b3db1c65f942f29152ce3008ba2c89" is_pre_election: true
I20260925 12:23:12.896649 12742 raft_consensus.cc:2454] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a17551c468e4abda8e7cbeb840ab68b in term 0.
I20260925 12:23:12.896909 12538 leader_election.cc:302] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b3db1c65f942f29152ce3008ba2c89, 2a17551c468e4abda8e7cbeb840ab68b; no voters:
I20260925 12:23:12.897060 12949 raft_consensus.cc:2781] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:23:12.897104 12949 raft_consensus.cc:482] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:23:12.897126 12949 raft_consensus.cc:3037] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:23:12.898123 12949 raft_consensus.cc:504] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.898299 12949 leader_election.cc:288] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [CANDIDATE]: Term 1 election: Requested vote from peers 02b3db1c65f942f29152ce3008ba2c89 (127.1.47.130:37287), 3acbc50d382e4feca4bcd69a66a02371 (127.1.47.131:38663)
I20260925 12:23:12.898553 12742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b348148259c64a7f934ec2d2097c8ef7" candidate_uuid: "2a17551c468e4abda8e7cbeb840ab68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02b3db1c65f942f29152ce3008ba2c89"
I20260925 12:23:12.898643 12742 raft_consensus.cc:3037] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:23:12.899605 12742 raft_consensus.cc:2454] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a17551c468e4abda8e7cbeb840ab68b in term 1.
I20260925 12:23:12.899811 12538 leader_election.cc:302] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b3db1c65f942f29152ce3008ba2c89, 2a17551c468e4abda8e7cbeb840ab68b; no voters:
I20260925 12:23:12.899822 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.886885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:42795 (local address 127.1.47.131:38663)
0925 12:23:12.886971 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.886973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.896323 (+ 9350us) server_negotiation.cc:407] Connection header received
0925 12:23:12.896418 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.896419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.896431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.896458 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:23:12.896682 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.897383 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.898903 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.899435 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.899480 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.899489 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.899715 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.899756 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 12:23:12.899914 12949 raft_consensus.cc:2781] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:23:12.900077 12949 raft_consensus.cc:686] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [term 1 LEADER]: Becoming Leader. State: Replica: 2a17551c468e4abda8e7cbeb840ab68b, State: Running, Role: LEADER
I20260925 12:23:12.900182 12949 consensus_queue.cc:237] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } }
I20260925 12:23:12.900264 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.886731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38663 (local address 127.1.47.129:42795)
0925 12:23:12.896275 (+ 9544us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.896286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:23:12.896353 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:23:12.896514 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:23:12.896515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:23:12.896516 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:23:12.896546 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.896547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.897620 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.897620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.898806 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:23:12.898809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.899540 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.899541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:23:12.899596 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.899599 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:23:12.900084 (+ 485us) client_negotiation.cc:240] Negotiation successful
0925 12:23:12.900160 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9530,"thread_start_us":105,"threads_started":1}
I20260925 12:23:12.901135 12438 catalog_manager.cc:5495] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a17551c468e4abda8e7cbeb840ab68b (127.1.47.129). New cstate: current_term: 1 leader_uuid: "2a17551c468e4abda8e7cbeb840ab68b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a17551c468e4abda8e7cbeb840ab68b" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38555 } health_report { overall_health: HEALTHY } } }
I20260925 12:23:12.901778 12876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b348148259c64a7f934ec2d2097c8ef7" candidate_uuid: "2a17551c468e4abda8e7cbeb840ab68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbc50d382e4feca4bcd69a66a02371"
I20260925 12:23:12.901914 12876 raft_consensus.cc:3037] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:23:12.902822 12876 raft_consensus.cc:2454] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a17551c468e4abda8e7cbeb840ab68b in term 1.
I20260925 12:23:12.901779 12877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b348148259c64a7f934ec2d2097c8ef7" candidate_uuid: "2a17551c468e4abda8e7cbeb840ab68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbc50d382e4feca4bcd69a66a02371" is_pre_election: true
I20260925 12:23:12.903095 12877 raft_consensus.cc:2362] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2a17551c468e4abda8e7cbeb840ab68b in term 1. Re-sending same reply.
I20260925 12:23:12.917518 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.913993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.1.47.129:38555)
0925 12:23:12.914050 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:12.914053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:12.914061 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:23:12.914102 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:12.914103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:12.914131 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:12.914217 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:23:12.914391 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.915174 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.916377 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:12.916612 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:12.916705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:12.916896 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:23:12.917048 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:23:12.917141 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:12.917411 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 12:23:12.917434 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:23:12.920219 12742 raft_consensus.cc:1261] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 1 FOLLOWER]: Refusing update from remote peer 2a17551c468e4abda8e7cbeb840ab68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:23:12.920356 12877 raft_consensus.cc:1261] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 1 FOLLOWER]: Refusing update from remote peer 2a17551c468e4abda8e7cbeb840ab68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:23:12.920581 12949 consensus_queue.cc:1035] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "02b3db1c65f942f29152ce3008ba2c89" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:23:12.920749 12949 consensus_queue.cc:1035] T b348148259c64a7f934ec2d2097c8ef7 P 2a17551c468e4abda8e7cbeb840ab68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acbc50d382e4feca4bcd69a66a02371" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:23:12.925323 12955 mvcc.cc:205] Tried to move back new op lower bound from 7333228514998149120 to 7333228514919813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333228514998149120}]
I20260925 12:23:12.927560 12957 mvcc.cc:205] Tried to move back new op lower bound from 7333228514998149120 to 7333228514919813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333228514998149120}]
I20260925 12:23:12.928097 12956 mvcc.cc:205] Tried to move back new op lower bound from 7333228514998149120 to 7333228514919813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333228514998149120}]
I20260925 12:23:12.935120 12439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:23:12.935241 12439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:23:12.936075 12439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9784e1adfaf34b7690da4bc0eec5828e: Sending DeleteTablet for 3 replicas of tablet b348148259c64a7f934ec2d2097c8ef7
I20260925 12:23:12.936498 12857 tablet_service.cc:1514] Processing DeleteTablet for tablet b348148259c64a7f934ec2d2097c8ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:23:12 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:57942
I20260925 12:23:12.936937 12722 tablet_service.cc:1514] Processing DeleteTablet for tablet b348148259c64a7f934ec2d2097c8ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:23:12 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:51442
I20260925 12:23:12.937388 12574 tablet_service.cc:1514] Processing DeleteTablet for tablet b348148259c64a7f934ec2d2097c8ef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:23:12 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56936
I20260925 12:23:12.937911 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 12520
I20260925 12:23:12.938993 12966 tablet_replica.cc:330] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: stopping tablet replica
I20260925 12:23:12.939179 12966 raft_consensus.cc:2229] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:23:12.939590 12966 raft_consensus.cc:2258] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:23:12.940016 12965 tablet_replica.cc:330] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: stopping tablet replica
I20260925 12:23:12.940163 12965 raft_consensus.cc:2229] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:23:12.940204 12966 ts_tablet_manager.cc:1905] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:23:12.940356 12965 raft_consensus.cc:2258] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:23:12.940837 12965 ts_tablet_manager.cc:1905] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:23:12.941944 12966 ts_tablet_manager.cc:1918] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:23:12.942065 12966 log.cc:1196] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/b348148259c64a7f934ec2d2097c8ef7
I20260925 12:23:12.942456 12965 ts_tablet_manager.cc:1918] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:23:12.942580 12965 log.cc:1196] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/b348148259c64a7f934ec2d2097c8ef7
I20260925 12:23:12.942694 12966 ts_tablet_manager.cc:1939] T b348148259c64a7f934ec2d2097c8ef7 P 02b3db1c65f942f29152ce3008ba2c89: Deleting consensus metadata
I20260925 12:23:12.943154 12425 catalog_manager.cc:4841] TS 02b3db1c65f942f29152ce3008ba2c89 (127.1.47.130:37287): tablet b348148259c64a7f934ec2d2097c8ef7 (table test-table [id=41c4f09af1a748c8864469f85ac8b350]) successfully deleted
I20260925 12:23:12.943181 12965 ts_tablet_manager.cc:1939] T b348148259c64a7f934ec2d2097c8ef7 P 3acbc50d382e4feca4bcd69a66a02371: Deleting consensus metadata
I20260925 12:23:12.943638 12424 catalog_manager.cc:4841] TS 3acbc50d382e4feca4bcd69a66a02371 (127.1.47.131:38663): tablet b348148259c64a7f934ec2d2097c8ef7 (table test-table [id=41c4f09af1a748c8864469f85ac8b350]) successfully deleted
W20260925 12:23:12.947682 12426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:38555 (error 108)
I20260925 12:23:12.948560 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:12.948444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38555 (local address 127.0.0.1:56950)
0925 12:23:12.948493 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 12:23:12.948525 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 12:23:12.948789 12426 catalog_manager.cc:4584] TS 2a17551c468e4abda8e7cbeb840ab68b (127.1.47.129:38555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b348148259c64a7f934ec2d2097c8ef7: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38555: connect: Connection refused (error 111)
I20260925 12:23:12.949030 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 12657
I20260925 12:23:12.955492 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 12794
I20260925 12:23:12.961869 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 12407
I20260925 12:23:12.969480 1214 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 12:23:13.304962 1214 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 12:23:13.305071 1214 mini_ranger.cc:66] Stopping Ranger...
I20260925 12:23:13.650053 1214 mini_ranger.cc:68] Stopped Ranger
2026-09-25 12:23:13.650 UTC [11993] LOG: received smart shutdown request
2026-09-25 12:23:13.653 UTC [11993] LOG: background worker "logical replication launcher" (PID 12002) exited with exit code 1
2026-09-25 12:23:13.653 UTC [11997] LOG: shutting down
2026-09-25 12:23:13.827 UTC [11993] LOG: database system is shut down
2026-09-25T12:23:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47855 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:23:13 dist-test-slave-l9ww krb5kdc[12977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:23:13 dist-test-slave-l9ww krb5kdc[12977](info): set up 2 sockets
Sep 25 12:23:13 dist-test-slave-l9ww krb5kdc[12977](info): commencing operation
krb5kdc: starting...
W20260925 12:23:14.991840 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:23:15 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790338995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:23:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:23:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangeradmin_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangeradmin_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangerlookup_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangerlookup_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
I20260925 12:23:15.159340 1214 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuGajnc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/postgres -l logfile start
2026-09-25 12:23:17.762 UTC [13007] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 12:23:17.762 UTC [13007] LOG: listening on IPv4 address "127.1.47.148", port 56397
2026-09-25 12:23:17.766 UTC [13007] LOG: listening on Unix socket "/tmp/.s.PGSQL.56397"
2026-09-25 12:23:17.801 UTC [13010] LOG: database system was shut down at 2026-09-25 12:23:16 UTC
2026-09-25 12:23:17.805 UTC [13007] LOG: database system is ready to accept connections
I20260925 12:23:18.788113 1214 mini_postgres.cc:96] Postgres bound to 56397
2026-09-25 12:23:18.794 UTC [13018] FATAL: database "slave" does not exist
127.1.47.148:56397 - accepting connections
I20260925 12:23:18.794776 1214 mini_ranger.cc:161] Starting Ranger...
I20260925 12:23:18.809577 1214 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 12:23:19.367292 1214 mini_ranger.cc:87] Created ranger Postgres database
I20260925 12:23:19.367359 1214 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin
2026-09-25 12:23:19,654 [I] DB FLAVOR :POSTGRES
2026-09-25 12:23:19,655 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 12:23:19,655 [I] Checking connection..
2026-09-25 12:23:19,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 12:23:19,906 [I] Checking connection passed.
2026-09-25 12:23:19,907 [I] --------- Verifying version history table ---------
2026-09-25 12:23:19,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:23:20,144 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 12:23:20,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:23:20,403 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 12:23:20,404 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 12:23:20,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 12:23:20.644 UTC [13082] WARNING: there is no transaction in progress
2026-09-25 12:23:20,650 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 12:23:20,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 12:23:20,907 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 12:23:20,908 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 12:23:20,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 12:23:21,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 12:23:21,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l9ww.c.gcp-upstream.internal','N') ;"
2026-09-25 12:23:21,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 12:23:21,891 [I] Table x_portal_user does not exist in database ranger
2026-09-25 12:23:21,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 12:23:22,138 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 12:23:22,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 12:23:22,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 12:23:22,633 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 12:23:22,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 12:23:23.652 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.670 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.688 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.705 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.722 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.742 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.809 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.817 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.833 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.843 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.853 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.868 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.879 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.886 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.893 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:23.899 UTC [13190] WARNING: there is no transaction in progress
2026-09-25 12:23:24,641 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 12:23:24,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 12:23:24,896 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 12:23:24,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 12:23:25,144 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 12:23:25,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 12:23:25,380 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 12:23:25,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 12:23:25,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 12:23:25,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l9ww.c.gcp-upstream.internal';"
2026-09-25 12:23:26,167 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 12:23:26,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 12:23:26,404 [I] DB_PATCHES have already been applied
I20260925 12:23:26.410046 1214 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuGajnc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 12:23:26.410136 1214 mini_ranger.cc:193] Using host: 127.1.47.148
I20260925 12:23:26.413151 1214 mini_ranger.cc:227] Ranger admin URL: http://127.1.47.148:57865
Sep 25, 2026 12:23:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:23:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:23:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 12:23:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 12:23:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 12:23:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 12:23:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 12:23:27.529795 1214 mini_ranger.cc:160] Time spent starting Ranger: real 8.735s user 0.000s sys 0.008s
Sep 25, 2026 12:23:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 12:23:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 12:23:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 12:23:48 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 12:23:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 12:23:50.505995 1214 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangerkms_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/rangerkms_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/HTTP_127.1.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 12:23:50 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339030, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 12:23:50.593880 1214 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 12:23:50.603624 1214 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 12:23:51.581961 1214 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 12:23:51.582067 1214 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms
I20260925 12:23:51.582089 1214 mini_ranger_kms.cc:227] Using postgres at 127.1.47.148:56397
2026-09-25 12:23:51,744 [I] DB FLAVOR :POSTGRES
2026-09-25 12:23:51,744 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 12:23:51,744 [I] Checking connection
2026-09-25 12:23:51,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 12:23:51,998 [I] connection success
2026-09-25 12:23:51,998 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 12:23:51,998 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 12:23:51,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 12:23:52,236 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 12:23:52,236 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 12:23:52,236 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 12:23:52,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.47.148:56397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 12:23:52,495 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 12:23:52.998150 1214 mini_ranger_kms.cc:310] Created kms service
I20260925 12:23:53.218976 1214 mini_ranger_kms.cc:326] Created kudu user
I20260925 12:23:53.273437 1214 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 12:23:53 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339030, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:53.731775 1214 mini_ranger_kms.cc:384] Added ranger policy
I20260925 12:23:53.732005 1214 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuGajnc/build/release/bin/postgresql.jar:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuGajnc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 12:23:53.732029 1214 mini_ranger_kms.cc:240] Using host: 127.1.47.148
I20260925 12:23:53.735270 1214 mini_ranger_kms.cc:276] Ranger KMS PID: 13371
I20260925 12:23:53.735344 1214 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.47.148:41729
Sep 25, 2026 12:23:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:23:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:23:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 12:23:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 12:23:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 12:23:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 12:23:56.051899 1214 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.458s user 0.001s sys 0.144s
I20260925 12:23:56.052016 1214 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 12:23:56.052066 1214 mini_ranger_kms.cc:401] 127.1.47.148:41729/kms/v1/keys
Sep 25 12:23:56 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790339036, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 12:23:56 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339036, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
Sep 25 12:23:57 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339036, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.47.148@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
Sep 25 12:23:59 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:23:59.388013 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35793
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:35793
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:23:59.471513 13421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:23:59.471938 13421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:23:59.471976 13421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:23:59.473290 13421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:23:59.473340 13421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:23:59.473356 13421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:23:59.473371 13421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:23:59.473426 13421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:23:59.475085 13421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.47.148:41729/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:35793
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:23:59.475344 13421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:23:59.475607 13421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:23:59.480751 13427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:23:59.480795 13429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:59.480821 13421 server_base.cc:1034] running on GCE node
W20260925 12:23:59.480767 13426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:23:59.481328 13421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:23:59.481609 13421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:23:59.482749 13421 hybrid_clock.cc:648] HybridClock initialized: now 1790339039482737 us; error 29 us; skew 500 ppm
Sep 25 12:23:59 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:23:59.485301 13421 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:23:59.485757 13421 webserver.cc:460] Webserver started at http://127.1.47.190:33249/ using document root <none> and password file <none>
I20260925 12:23:59.485994 13421 fs_manager.cc:359] Metadata directory not provided
I20260925 12:23:59.486042 13421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:23:59.486179 13421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:23:59 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:23:59.548466 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "dfbf636c588e46e7bae651cbceff45e7"
format_stamp: "Formatted at 2026-09-25 12:23:59 on dist-test-slave-l9ww"
server_key: "b953ffeb6e598e2cd016aac0ca799324"
server_key_iv: "cc8079ae56dfe5915278be4d747afdb4"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:59.548923 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "dfbf636c588e46e7bae651cbceff45e7"
format_stamp: "Formatted at 2026-09-25 12:23:59 on dist-test-slave-l9ww"
server_key: "b953ffeb6e598e2cd016aac0ca799324"
server_key_iv: "cc8079ae56dfe5915278be4d747afdb4"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:59.550869 13421 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 12:23:59.563282 13436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:23:59.563611 13421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:23:59.563732 13421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "dfbf636c588e46e7bae651cbceff45e7"
format_stamp: "Formatted at 2026-09-25 12:23:59 on dist-test-slave-l9ww"
server_key: "b953ffeb6e598e2cd016aac0ca799324"
server_key_iv: "cc8079ae56dfe5915278be4d747afdb4"
server_key_version: "kuduclusterkey@0"
I20260925 12:23:59.563827 13421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:23:59.592808 13421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:23:59.595153 13421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:23:59.595314 13421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:23:59.600320 13421 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:35793
I20260925 12:23:59.600348 13488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:35793 every 8 connection(s)
I20260925 12:23:59.600806 13421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:23:59.601544 13489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:23:59.603818 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Bootstrap starting.
I20260925 12:23:59.604391 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 13421
I20260925 12:23:59.604485 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:23:59.604519 13489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Neither blocks nor log segments found. Creating new log.
I20260925 12:23:59.604660 1214 external_mini_cluster.cc:1428] Setting key 9379d5c14473a406fa3c80eae053b90e
I20260925 12:23:59.604902 13489 log.cc:824] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:23:59.605764 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: No bootstrap required, opened a new log
I20260925 12:23:59.608207 13489 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:23:59.608376 13489 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:23:59.608415 13489 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfbf636c588e46e7bae651cbceff45e7, State: Initialized, Role: FOLLOWER
I20260925 12:23:59.608579 13489 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:23:59.608656 13489 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:23:59.608731 13489 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:23:59.608790 13489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:23:59.609517 13489 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:23:59.609640 13489 leader_election.cc:302] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfbf636c588e46e7bae651cbceff45e7; no voters:
I20260925 12:23:59.609824 13489 leader_election.cc:288] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:23:59.609877 13494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:23:59.610109 13494 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 LEADER]: Becoming Leader. State: Replica: dfbf636c588e46e7bae651cbceff45e7, State: Running, Role: LEADER
I20260925 12:23:59.610179 13489 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:23:59.610262 13494 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:23:59.610647 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfbf636c588e46e7bae651cbceff45e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } } }
I20260925 12:23:59.610770 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: This master's current role is: LEADER
I20260925 12:23:59.610939 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfbf636c588e46e7bae651cbceff45e7. Latest consensus state: current_term: 1 leader_uuid: "dfbf636c588e46e7bae651cbceff45e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } } }
I20260925 12:23:59.611030 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: This master's current role is: LEADER
I20260925 12:23:59.612042 13500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:23:59.612572 13500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 12:23:59 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:23:59.613657 13489 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 12:23:59.618291 13500 catalog_manager.cc:1347] Generated new cluster ID: c404879dc6ac4f73a2b264e40b6aea93
I20260925 12:23:59.618417 13500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:23:59.621376 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:23:59.605316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.1.47.190:35793)
0925 12:23:59.605514 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 12:23:59.605523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:23:59.605554 (+ 31us) server_negotiation.cc:407] Connection header received
0925 12:23:59.606150 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:23:59.606160 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:23:59.606233 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:23:59.606383 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:23:59.607161 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:59.608629 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:59.609794 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:23:59.610438 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:23:59.610881 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:23:59.613678 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:23:59.613681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:23:59.613681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:23:59.613692 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:23:59.619314 (+ 5622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:59.619804 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:59.619807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:59.619807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:59.619938 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:23:59.620119 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:23:59.620119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:23:59.620120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:23:59.620419 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:23:59.620544 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 12:23:59.621022 (+ 478us) server_negotiation.cc:299] Negotiation successful
0925 12:23:59.621168 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260925 12:23:59.639458 13500 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:23:59.640373 13500 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:23:59.649407 13500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Generated new TSK 0
I20260925 12:23:59.649708 13500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:24:00.377334 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:00.477404 13534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:00.477669 13534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:00.477711 13534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:00.479100 13534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:00.479183 13534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:00.479233 13534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:00.480940 13534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:00.481241 13534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:00.481506 13534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:00.486693 13540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:00.486693 13542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:00.487020 13539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:00.488000 13534 server_base.cc:1034] running on GCE node
I20260925 12:24:00.488237 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:00.488507 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:00.489668 13534 hybrid_clock.cc:648] HybridClock initialized: now 1790339040489641 us; error 44 us; skew 500 ppm
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:00.492390 13534 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:00.492815 13534 webserver.cc:460] Webserver started at http://127.1.47.129:41045/ using document root <none> and password file <none>
I20260925 12:24:00.493080 13534 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:00.493152 13534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:00.493311 13534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:00.507517 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "c051256f32d4440b8bef5bf67b3dfd81"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "77594f6e991b9642169d47fb1935f2b7"
server_key_iv: "e8b84956c0c3a168d808a8e9261b78e5"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.508044 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "c051256f32d4440b8bef5bf67b3dfd81"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "77594f6e991b9642169d47fb1935f2b7"
server_key_iv: "e8b84956c0c3a168d808a8e9261b78e5"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.510758 13534 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:00.531560 13549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.531808 13534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 12:24:00.531913 13534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "c051256f32d4440b8bef5bf67b3dfd81"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "77594f6e991b9642169d47fb1935f2b7"
server_key_iv: "e8b84956c0c3a168d808a8e9261b78e5"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.532011 13534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:00.561812 13534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:00.564893 13534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:00.565058 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:00.565275 13534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:00.565618 13534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:00.565644 13534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.565667 13534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:00.565680 13534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.572777 13534 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:37365
I20260925 12:24:00.573311 13534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:00.577257 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 13534
I20260925 12:24:00.577343 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:24:00.577548 1214 external_mini_cluster.cc:1428] Setting key 5d736544b331bc683cb76dd1331fd89d
I20260925 12:24:00.581894 13664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:37365 every 8 connection(s)
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
I20260925 12:24:00.597718 13668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:00.581577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.129:33873)
0925 12:24:00.581801 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:00.581862 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:00.582496 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:00.583834 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:00.583839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:00.584131 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:00.586777 (+ 2646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:00.586785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.587891 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.587892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:00.589462 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:00.589466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.590069 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.590069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:00.590218 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:00.592007 (+ 1789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:00.592018 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:00.593780 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:00.596037 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:00.596045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:00.596048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:00.596364 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:00.596594 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:00.596594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:00.596595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:00.596729 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:00.597009 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:00.597013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:00.597192 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 12:24:00.597294 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 12:24:00.597459 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260925 12:24:00.598286 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:00.581616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33873 (local address 127.1.47.190:35793)
0925 12:24:00.583556 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:00.583558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:00.583568 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:24:00.583597 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:00.583597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:00.583612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:00.583699 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:00.586939 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.587727 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.589576 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.589884 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.589953 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:00.593917 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:00.593919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:00.593920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:00.593928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:00.595934 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:00.596450 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:00.596451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:00.596451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:00.596517 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:00.596793 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:00.596794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:00.596794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:00.596931 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:00.596985 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:00.598085 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0925 12:24:00.598182 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1907,"thread_start_us":104,"threads_started":1}
I20260925 12:24:00.598827 13665 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:00.598958 13665 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:00.599172 13665 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:00.599788 13453 ts_manager.cc:194] Registered new tserver with Master: c051256f32d4440b8bef5bf67b3dfd81 (127.1.47.129:37365)
I20260925 12:24:00.600680 13453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:33873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:24:00.620795 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:00.739001 13671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:00.739359 13671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:00.739471 13671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:00.741318 13671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:00.741382 13671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:00.741425 13671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:00.743722 13671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:00.744027 13671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:00.744359 13671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:00.750766 13679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:00.750830 13681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:00.751130 13678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:00.751361 13671 server_base.cc:1034] running on GCE node
I20260925 12:24:00.751596 13671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:00.751909 13671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:00.755213 13671 hybrid_clock.cc:648] HybridClock initialized: now 1790339040755187 us; error 37 us; skew 500 ppm
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:00.758708 13671 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:00.759303 13671 webserver.cc:460] Webserver started at http://127.1.47.130:33831/ using document root <none> and password file <none>
I20260925 12:24:00.759593 13671 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:00.759665 13671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:00.759816 13671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:00.779281 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "9227960de2e548e4b7b507a503c6b88e"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "439dbbea7af7fb8b7d7840d2d7924830"
server_key_iv: "c617ba8c83adef7744f9c5708182f226"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.779810 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "9227960de2e548e4b7b507a503c6b88e"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "439dbbea7af7fb8b7d7840d2d7924830"
server_key_iv: "c617ba8c83adef7744f9c5708182f226"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.787694 13671 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260925 12:24:00.800518 13688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.801388 13671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 12:24:00.801563 13671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "9227960de2e548e4b7b507a503c6b88e"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "439dbbea7af7fb8b7d7840d2d7924830"
server_key_iv: "c617ba8c83adef7744f9c5708182f226"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:00.801751 13671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:00.843393 13671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:00.845985 13671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:00.846194 13671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:00.846498 13671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:00.846911 13671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:00.847000 13671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.847072 13671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:00.847127 13671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:00.854184 13671 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:45853
I20260925 12:24:00.854737 13671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:00.858634 13802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:45853 every 8 connection(s)
I20260925 12:24:00.861649 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 13671
I20260925 12:24:00.861769 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:24:00.861974 1214 external_mini_cluster.cc:1428] Setting key 69b791c050ddd1a157526af8fdb8621a
Sep 25 12:24:00 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
I20260925 12:24:00.871778 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:00.859983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:48793 (local address 127.1.47.190:35793)
0925 12:24:00.860054 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:00.860056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:00.860654 (+ 598us) server_negotiation.cc:407] Connection header received
0925 12:24:00.861384 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:00.861385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:00.861401 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:00.861433 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:00.862533 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.863251 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.864877 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.865214 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.865261 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:00.868146 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:00.868147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:00.868148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:00.868156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:00.870034 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:00.870577 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:00.870579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:00.870579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:00.870636 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:00.870937 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:00.870938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:00.870938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:00.871093 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:00.871154 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:00.871562 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 12:24:00.871670 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 12:24:00.872494 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:00.860099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.130:48793)
0925 12:24:00.860521 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:00.860568 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:00.861257 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:00.861651 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:00.861655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:00.862115 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:00.862407 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:00.862414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.863367 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.863368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:00.864769 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:00.864772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:00.865366 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:00.865367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:00.865519 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:00.866130 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:00.866140 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:00.867990 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:00.870140 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:00.870148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:00.870150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:00.870481 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:00.870709 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:00.870710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:00.870710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:00.870854 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:00.871192 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:00.871198 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:00.871384 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 12:24:00.872154 (+ 770us) client_negotiation.cc:240] Negotiation successful
0925 12:24:00.872310 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":108,"threads_started":1}
I20260925 12:24:00.873062 13803 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:00.873188 13803 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:00.873404 13803 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:00.873842 13453 ts_manager.cc:194] Registered new tserver with Master: 9227960de2e548e4b7b507a503c6b88e (127.1.47.130:45853)
I20260925 12:24:00.874323 13453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:48793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:24:00.895262 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:01.013937 13808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:01.014328 13808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:01.014436 13808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:01.016390 13808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:01.016513 13808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:01.016606 13808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:01.019001 13808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:01.019515 13808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:01.020117 13808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:01.027730 13813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.027731 13814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.032377 13808 server_base.cc:1034] running on GCE node
W20260925 12:24:01.033347 13816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.033677 13808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:01.034021 13808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:01.037322 13808 hybrid_clock.cc:648] HybridClock initialized: now 1790339041037985 us; error 768 us; skew 500 ppm
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:01.041272 13808 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:01.041939 13808 webserver.cc:460] Webserver started at http://127.1.47.131:37889/ using document root <none> and password file <none>
I20260925 12:24:01.042280 13808 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:01.042408 13808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:01.042591 13808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:01.058358 13808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "681c966183e1482789f82cfa6bc69ef3"
format_stamp: "Formatted at 2026-09-25 12:24:01 on dist-test-slave-l9ww"
server_key: "f0cfe40bfba09d0ce395bc9592831753"
server_key_iv: "9bc0d7ef7a78cdd477dc38fa9f89b633"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.058877 13808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "681c966183e1482789f82cfa6bc69ef3"
format_stamp: "Formatted at 2026-09-25 12:24:01 on dist-test-slave-l9ww"
server_key: "f0cfe40bfba09d0ce395bc9592831753"
server_key_iv: "9bc0d7ef7a78cdd477dc38fa9f89b633"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.060868 13808 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260925 12:24:01.074918 13823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.075202 13808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.075295 13808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "681c966183e1482789f82cfa6bc69ef3"
format_stamp: "Formatted at 2026-09-25 12:24:01 on dist-test-slave-l9ww"
server_key: "f0cfe40bfba09d0ce395bc9592831753"
server_key_iv: "9bc0d7ef7a78cdd477dc38fa9f89b633"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.075387 13808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:01.102010 13808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:01.104831 13808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:01.105105 13808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:01.105448 13808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:01.105949 13808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:01.106051 13808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.106132 13808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:01.106192 13808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.113863 13808 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:41589
I20260925 12:24:01.117890 13808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:24:01.117324 13936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:41589 every 8 connection(s)
I20260925 12:24:01.125015 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 13808
I20260925 12:24:01.125128 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:24:01.125329 1214 external_mini_cluster.cc:1428] Setting key dae5ce21d18ab726c9bf96bfb8a93d79
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:01.133421 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.119656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54679 (local address 127.1.47.190:35793)
0925 12:24:01.119707 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:01.119709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:01.119962 (+ 253us) server_negotiation.cc:407] Connection header received
0925 12:24:01.120833 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:01.120835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:01.120849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:01.120885 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:01.121847 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.122711 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.124372 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.124692 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.124745 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.129368 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:01.129369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:01.129370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:01.129379 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:01.131512 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.132137 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.132138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.132139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.132208 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.132545 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.132546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.132546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.132704 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:01.132764 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:01.133203 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 12:24:01.133309 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:24:01.134398 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.119350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.131:54679)
0925 12:24:01.119784 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:01.119848 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:01.120679 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:01.121095 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:01.121100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:01.121390 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:01.121702 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.121710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.122857 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.122858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.124255 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.124258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.126329 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.126330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.126495 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.127142 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:01.127155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:01.129228 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:01.131620 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.131628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.131631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.132035 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.132288 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.132289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.132289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.132444 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.132803 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:01.132810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:01.133017 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 12:24:01.133905 (+ 888us) client_negotiation.cc:240] Negotiation successful
0925 12:24:01.134127 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":96,"threads_started":1}
I20260925 12:24:01.135303 13937 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:01.135412 13937 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.135623 13937 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:01.136081 13453 ts_manager.cc:194] Registered new tserver with Master: 681c966183e1482789f82cfa6bc69ef3 (127.1.47.131:41589)
I20260925 12:24:01.136653 13453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:54679
I20260925 12:24:01.138384 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:01.139125 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 13534
I20260925 12:24:01.145937 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 13671
I20260925 12:24:01.152079 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 13808
I20260925 12:24:01.159353 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 13421
I20260925 12:24:01.166229 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35793
--webserver_interface=127.1.47.190
--webserver_port=33249
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:35793
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:01.248240 13940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:01.248492 13940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:01.248539 13940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:01.249732 13940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:01.249814 13940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:01.249837 13940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:01.249871 13940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:01.249918 13940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:01.251528 13940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:35793
--ranger_config_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:01.251817 13940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:01.252094 13940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:01.256866 13945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.256879 13946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.257077 13940 server_base.cc:1034] running on GCE node
W20260925 12:24:01.257162 13948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.257592 13940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:01.257942 13940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:01.259122 13940 hybrid_clock.cc:648] HybridClock initialized: now 1790339041259105 us; error 33 us; skew 500 ppm
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:01.262128 13940 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:01.262601 13940 webserver.cc:460] Webserver started at http://127.1.47.190:33249/ using document root <none> and password file <none>
I20260925 12:24:01.262841 13940 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:01.262919 13940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:01.337339 13940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.338246 13956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.338500 13940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.338613 13940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "dfbf636c588e46e7bae651cbceff45e7"
format_stamp: "Formatted at 2026-09-25 12:23:59 on dist-test-slave-l9ww"
server_key: "b953ffeb6e598e2cd016aac0ca799324"
server_key_iv: "cc8079ae56dfe5915278be4d747afdb4"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.338985 13940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:01.346552 13940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:01.348740 13940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:01.348913 13940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:01.353418 13940 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:35793
I20260925 12:24:01.353456 14008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:35793 every 8 connection(s)
I20260925 12:24:01.353904 13940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:01.355204 14009 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:24:01.355930 14009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Bootstrap starting.
I20260925 12:24:01.363262 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 13940
I20260925 12:24:01.363627 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:37365
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:35793
--webserver_port=41045
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:01.364296 14009 log.cc:824] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:01.366762 14009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:01.367087 14009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Bootstrap complete.
I20260925 12:24:01.369645 14009 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:24:01.369892 14009 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfbf636c588e46e7bae651cbceff45e7, State: Initialized, Role: FOLLOWER
I20260925 12:24:01.370069 14009 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:24:01.370141 14009 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:01.370203 14009 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:01.370249 14009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:24:01.371272 14009 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:24:01.371359 14009 leader_election.cc:302] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfbf636c588e46e7bae651cbceff45e7; no voters:
I20260925 12:24:01.371565 14009 leader_election.cc:288] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 12:24:01.371609 14013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 12:24:01.371726 14013 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [term 2 LEADER]: Becoming Leader. State: Replica: dfbf636c588e46e7bae651cbceff45e7, State: Running, Role: LEADER
I20260925 12:24:01.371837 14009 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:01.371817 14013 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } }
I20260925 12:24:01.372144 14015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfbf636c588e46e7bae651cbceff45e7. Latest consensus state: current_term: 2 leader_uuid: "dfbf636c588e46e7bae651cbceff45e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } } }
I20260925 12:24:01.372385 14015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:01.372416 14014 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dfbf636c588e46e7bae651cbceff45e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf636c588e46e7bae651cbceff45e7" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 35793 } } }
I20260925 12:24:01.372505 14014 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:01.372746 14021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:01.373338 14021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:01.373492 14021 catalog_manager.cc:1259] Loaded cluster ID: c404879dc6ac4f73a2b264e40b6aea93
I20260925 12:24:01.373538 14021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:01.373741 14009 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 12:24:01.376758 14021 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:01.377137 14021 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Loaded TSK: 0
I20260925 12:24:01.377382 14021 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 12:24:01.478570 14012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:01.478839 14012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:01.478879 14012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:01.480547 14012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:01.480609 14012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:01.480651 14012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:01.482776 14012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:37365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=41045
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:01.483022 14012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:01.483280 14012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:01.489826 14049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.489833 14047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.490192 14046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.490238 14012 server_base.cc:1034] running on GCE node
I20260925 12:24:01.490496 14012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:01.490751 14012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:01.491940 14012 hybrid_clock.cc:648] HybridClock initialized: now 1790339041491911 us; error 44 us; skew 500 ppm
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:01.498993 14012 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:01.499599 14012 webserver.cc:460] Webserver started at http://127.1.47.129:41045/ using document root <none> and password file <none>
I20260925 12:24:01.499989 14012 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:01.500067 14012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:01.520324 14012 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260925 12:24:01.521422 14057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.521598 14012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.521679 14012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "c051256f32d4440b8bef5bf67b3dfd81"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "77594f6e991b9642169d47fb1935f2b7"
server_key_iv: "e8b84956c0c3a168d808a8e9261b78e5"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.522143 14012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:01.547363 14012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:01.549942 14012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:01.550120 14012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:01.550390 14012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:01.550781 14012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:01.550827 14012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.550861 14012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:01.550880 14012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.558084 14012 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:37365
I20260925 12:24:01.558602 14012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:01.560696 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14012
I20260925 12:24:01.560971 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:45853
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:35793
--webserver_port=33831
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:01.560748 14170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:37365 every 8 connection(s)
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:01.579634 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.561392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.129:48893)
0925 12:24:01.562417 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:01.562463 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:01.563112 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:01.563892 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:01.563897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:01.564141 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:01.564432 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.564438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.566181 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.566182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.567402 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.567405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.568415 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.568416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.568603 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.570753 (+ 2150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:01.570764 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:01.572589 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:01.577507 (+ 4918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.577517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.577520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.577869 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.578188 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.578189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.578189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.578378 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.578912 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:01.578918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:01.579079 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 12:24:01.579194 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 12:24:01.579366 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"thread_start_us":84,"threads_started":1}
I20260925 12:24:01.580204 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.561946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:48893 (local address 127.1.47.190:35793)
0925 12:24:01.563326 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:01.563337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:01.563374 (+ 37us) server_negotiation.cc:407] Connection header received
0925 12:24:01.563518 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:01.563538 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:01.563592 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:01.563719 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:01.564620 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.566055 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.567508 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.568179 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.568295 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.572737 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:01.572742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:01.572742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:01.572757 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:01.577364 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.577976 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.577980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.577981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.578108 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.578462 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.578463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.578464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.578795 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:01.578880 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:01.579881 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0925 12:24:01.580008 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":80,"threads_started":1}
I20260925 12:24:01.580992 14171 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:01.581100 14171 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.581310 14171 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.582165 14171 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 12:24:01.582244 14171 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.582314 14171 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:01.582675 14171 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.582758 14171 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.583041 14171 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:24:01.677227 14175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:01.677438 14175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:01.677464 14175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:01.678788 14175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:01.678851 14175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:01.678896 14175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:01.680460 14175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:45853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=33831
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:01.680764 14175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:01.681000 14175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:01.687057 14181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.687135 14183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.687211 14175 server_base.cc:1034] running on GCE node
W20260925 12:24:01.687337 14180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.687530 14175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:01.687780 14175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:01.688932 14175 hybrid_clock.cc:648] HybridClock initialized: now 1790339041688908 us; error 56 us; skew 500 ppm
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:01.691645 14175 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:01.692216 14175 webserver.cc:460] Webserver started at http://127.1.47.130:33831/ using document root <none> and password file <none>
I20260925 12:24:01.692466 14175 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:01.692557 14175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:01.714785 14175 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.715758 14191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.716019 14175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:01.716120 14175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "9227960de2e548e4b7b507a503c6b88e"
format_stamp: "Formatted at 2026-09-25 12:24:00 on dist-test-slave-l9ww"
server_key: "439dbbea7af7fb8b7d7840d2d7924830"
server_key_iv: "c617ba8c83adef7744f9c5708182f226"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.716530 14175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:01.737887 14175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:01.740478 14175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:01.740674 14175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:01.740968 14175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:01.741411 14175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:01.741477 14175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.741524 14175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:01.741554 14175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.749626 14175 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:45853
I20260925 12:24:01.750074 14175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:01.750692 14304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:45853 every 8 connection(s)
I20260925 12:24:01.753275 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14175
I20260925 12:24:01.753577 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.47.148:41729/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:41589
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:35793
--webserver_port=37889
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:01.761790 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.751359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:39167 (local address 127.1.47.190:35793)
0925 12:24:01.751410 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:01.751413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:01.751425 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:24:01.752055 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:01.752057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:01.752074 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:01.752166 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:01.752958 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.753766 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.755456 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.755805 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.755851 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.758483 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:01.758485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:01.758485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:01.758494 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:01.760181 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.760698 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.760699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.760700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.760764 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.761062 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.761063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.761063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.761193 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:01.761278 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:01.761591 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 12:24:01.761686 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:24:01.762228 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.750977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.130:39167)
0925 12:24:01.751168 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:01.751232 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:01.751923 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:01.752252 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:01.752258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:01.752516 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:01.752825 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.752832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.753900 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.753901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.755334 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.755338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.756025 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.756026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.756187 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.756868 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:01.756878 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:01.758362 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:01.760313 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.760321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.760324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.760595 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.760847 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.760848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.760849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.760987 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.761268 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:01.761275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:01.761442 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 12:24:01.761570 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 12:24:01.761736 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260925 12:24:01.762734 14305 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:01.762872 14305 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.763074 14305 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.763839 14305 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 12:24:01.763964 14305 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.764045 14305 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:01.764359 14305 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.764439 14305 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.764717 14305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:24:01.876427 14308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:01.876658 14308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:01.876698 14308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:01.877960 14308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:01.878029 14308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:01.878077 14308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:01.879689 14308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.47.148:41729/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:41589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=37889
--tserver_master_addrs=127.1.47.190:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:01.879948 14308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:01.880228 14308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:01.885632 14315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:01.885635 14314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.886402 14308 server_base.cc:1034] running on GCE node
W20260925 12:24:01.887181 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:01.887463 14308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:01.887769 14308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:01.888955 14308 hybrid_clock.cc:648] HybridClock initialized: now 1790339041888939 us; error 33 us; skew 500 ppm
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:01.894081 14308 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:01.894635 14308 webserver.cc:460] Webserver started at http://127.1.47.131:37889/ using document root <none> and password file <none>
I20260925 12:24:01.894953 14308 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:01.895033 14308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:01.912684 14308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:01.913820 14325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.914050 14308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:01.914142 14308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "681c966183e1482789f82cfa6bc69ef3"
format_stamp: "Formatted at 2026-09-25 12:24:01 on dist-test-slave-l9ww"
server_key: "f0cfe40bfba09d0ce395bc9592831753"
server_key_iv: "9bc0d7ef7a78cdd477dc38fa9f89b633"
server_key_version: "kuduclusterkey@0"
I20260925 12:24:01.914628 14308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:01.941725 14308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:01.945497 14308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:01.945672 14308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:01.945950 14308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:01.946317 14308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:01.946365 14308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.946396 14308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:01.946424 14308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:01.953348 14308 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:41589
I20260925 12:24:01.953540 14438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:41589 every 8 connection(s)
I20260925 12:24:01.953840 14308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:24:01 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339041, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:01.960888 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14308
I20260925 12:24:01.964359 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.954700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:54325 (local address 127.1.47.190:35793)
0925 12:24:01.954746 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:01.954748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:01.954961 (+ 213us) server_negotiation.cc:407] Connection header received
0925 12:24:01.955574 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:01.955576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:01.955590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:01.955657 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:01.956417 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.956929 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.958045 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.958249 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.958343 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.960898 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:01.960900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:01.960900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:01.960910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:01.962646 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.963177 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.963179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.963179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.963250 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.963568 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.963569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.963569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.963719 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:01.963799 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:01.964174 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 12:24:01.964262 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:24:01.964531 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.954664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:35793 (local address 127.1.47.131:54325)
0925 12:24:01.954826 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:01.954870 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:01.955456 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:01.955735 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:01.955739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:01.956032 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:01.956313 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.956320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.957034 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.957035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.957954 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:01.957956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.958377 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.958379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:01.958495 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.959157 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:01.959166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:01.960777 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:01.962738 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.962744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.962747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.963071 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.963336 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:01.963337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:01.963337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:01.963478 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:01.963803 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:01.963809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:01.964005 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 12:24:01.964126 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 12:24:01.964284 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260925 12:24:01.965252 14439 heartbeater.cc:347] Connected to a master server at 127.1.47.190:35793
I20260925 12:24:01.965364 14439 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.965581 14439 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.967279 14439 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 12:24:01.967353 14439 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.967742 14439 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:01.968137 14439 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:01.968215 14439 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
W20260925 12:24:01.968554 14439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 12:24:01.974269 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:01.961539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.1.47.190:35793)
0925 12:24:01.961662 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:01.961664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:01.961669 (+ 5us) server_negotiation.cc:407] Connection header received
0925 12:24:01.961686 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:01.961687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:01.961696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:01.961773 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:01.962555 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.963571 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.964624 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:01.964954 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:01.965030 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:01.966561 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:01.966563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:01.966563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:01.966570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:01.972644 (+ 6074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.973369 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.973370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.973371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.973458 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:01.973710 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:01.973711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:01.973712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:01.973835 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:01.974023 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:01.974059 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 12:24:01.974152 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
Sep 25 12:24:02 dist-test-slave-l9ww krb5kdc[12977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790339042, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:02.583241 14171 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:02.583448 14171 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:02.583947 13973 ts_manager.cc:194] Registered new tserver with Master: c051256f32d4440b8bef5bf67b3dfd81 (127.1.47.129:37365)
I20260925 12:24:02.584975 13973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:48893
I20260925 12:24:02.764941 14305 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:02.765113 14305 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:02.765437 13973 ts_manager.cc:194] Registered new tserver with Master: 9227960de2e548e4b7b507a503c6b88e (127.1.47.130:45853)
I20260925 12:24:02.765965 13973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:39167
Sep 25 12:24:02 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339042, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
I20260925 12:24:02.968726 14439 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:02.968905 14439 heartbeater.cc:511] Master 127.1.47.190:35793 requested a full tablet report, sending...
I20260925 12:24:02.969272 13973 ts_manager.cc:194] Registered new tserver with Master: 681c966183e1482789f82cfa6bc69ef3 (127.1.47.131:41589)
I20260925 12:24:02.969817 13973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:54325
I20260925 12:24:02.975469 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:02.984645 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.976472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.1.47.190:35793)
0925 12:24:02.976586 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:02.976589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:02.976651 (+ 62us) server_negotiation.cc:407] Connection header received
0925 12:24:02.976723 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:02.976724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:02.976736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:02.976820 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:02.977535 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.978316 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.979373 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.979675 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.979745 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:02.981408 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:02.981409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:02.981409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:02.981416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:02.983114 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:02.983482 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:02.983483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:02.983483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:02.983551 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:02.983742 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:02.983743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:02.983743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:02.983896 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:02.983985 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:02.984439 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 12:24:02.984541 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260925 12:24:02.987396 13973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:24:02.990089 13973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:24:02.999146 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.993654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:41589 (local address 127.0.0.1:47512)
0925 12:24:02.994291 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:02.994331 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:02.994398 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:02.995106 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:02.995107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:02.995109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:02.995157 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:02.995158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.996502 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.996503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:02.997865 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:02.997868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.998845 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.998846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:02.998942 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:02.998959 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 12:24:02.999008 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:24:02.999059 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":30,"thread_start_us":76,"threads_started":1}
I20260925 12:24:02.999902 14469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.993994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.1.47.129:37365)
0925 12:24:02.994524 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:02.994529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:02.994667 (+ 138us) server_negotiation.cc:407] Connection header received
0925 12:24:02.994704 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:02.994706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:02.994722 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:02.994795 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:02.994998 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.996185 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.997972 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.999098 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.999158 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:02.999489 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:02.999741 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 12:24:02.999789 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":48,"threads_started":1}
I20260925 12:24:03.000283 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.993654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:37365 (local address 127.0.0.1:57490)
0925 12:24:02.994571 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:02.994580 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:02.994619 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:02.994811 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:02.994815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:02.994817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:02.994894 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:02.994896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.996290 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.996291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:02.997863 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:02.997866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.999578 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.999579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:02.999642 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:02.999646 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:03.000155 (+ 509us) client_negotiation.cc:240] Negotiation successful
0925 12:24:03.000202 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260925 12:24:03.001180 14105 tablet_service.cc:1467] Processing CreateTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 (DEFAULT_TABLE table=test-table [id=d7765de68f444142b7de106dabbadf13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:03.001605 14105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1897aadd7e11474a8bbf4f8ec3a17903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:03.003481 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.993993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.1.47.131:41589)
0925 12:24:02.994917 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:02.994931 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:02.994940 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:24:02.994972 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:02.994974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:02.994988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:02.995229 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:02.995302 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.996396 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:02.997972 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:02.998742 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.003244 (+ 4502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.003326 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:03.003353 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 12:24:03.003400 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":73,"threads_started":1}
I20260925 12:24:03.004137 14354 tablet_service.cc:1467] Processing CreateTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 (DEFAULT_TABLE table=test-table [id=d7765de68f444142b7de106dabbadf13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:03.004509 14354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1897aadd7e11474a8bbf4f8ec3a17903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:03.008029 14473 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Bootstrap starting.
I20260925 12:24:03.008443 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.993683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:45853 (local address 127.0.0.1:52786)
0925 12:24:02.999325 (+ 5642us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:02.999335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:02.999374 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:02.999593 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:02.999594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:02.999595 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:02.999628 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:02.999629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.001095 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.001096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.002275 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:03.002278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.008214 (+ 5936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.008215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.008294 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.008298 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:03.008327 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 12:24:03.008374 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5631,"thread_start_us":83,"threads_started":1}
I20260925 12:24:03.008855 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:02.994130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.1.47.130:45853)
0925 12:24:02.994276 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:02.994281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:02.999440 (+ 5159us) server_negotiation.cc:407] Connection header received
0925 12:24:02.999484 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:02.999487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:02.999504 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:02.999691 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:02.999769 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.001003 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.002378 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.003131 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.008626 (+ 5495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.008708 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:03.008735 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 12:24:03.008784 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260925 12:24:03.009131 14473 tablet_bootstrap.cc:654] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:03.009449 14239 tablet_service.cc:1467] Processing CreateTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 (DEFAULT_TABLE table=test-table [id=d7765de68f444142b7de106dabbadf13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:03.009758 14239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1897aadd7e11474a8bbf4f8ec3a17903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:03.010459 14473 log.cc:824] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:03.012006 14472 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Bootstrap starting.
I20260925 12:24:03.012573 14473 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: No bootstrap required, opened a new log
I20260925 12:24:03.012646 14473 ts_tablet_manager.cc:1397] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 12:24:03.012705 14472 tablet_bootstrap.cc:654] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:03.013015 14472 log.cc:824] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:03.013451 14475 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Bootstrap starting.
I20260925 12:24:03.013993 14475 tablet_bootstrap.cc:654] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:03.014248 14475 log.cc:824] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:03.014885 14475 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: No bootstrap required, opened a new log
I20260925 12:24:03.014950 14475 ts_tablet_manager.cc:1397] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:24:03.015286 14473 raft_consensus.cc:348] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.015448 14473 raft_consensus.cc:374] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:03.015484 14473 raft_consensus.cc:729] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681c966183e1482789f82cfa6bc69ef3, State: Initialized, Role: FOLLOWER
I20260925 12:24:03.015609 14473 consensus_queue.cc:260] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.016494 14473 ts_tablet_manager.cc:1428] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 12:24:03.016897 14475 raft_consensus.cc:348] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.017068 14475 raft_consensus.cc:374] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:03.017098 14475 raft_consensus.cc:729] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9227960de2e548e4b7b507a503c6b88e, State: Initialized, Role: FOLLOWER
I20260925 12:24:03.017198 14475 consensus_queue.cc:260] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.017437 14475 ts_tablet_manager.cc:1428] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:24:03.017490 14305 heartbeater.cc:503] Master 127.1.47.190:35793 was elected leader, sending a full tablet report...
I20260925 12:24:03.017617 14439 heartbeater.cc:503] Master 127.1.47.190:35793 was elected leader, sending a full tablet report...
I20260925 12:24:03.024660 14472 tablet_bootstrap.cc:492] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: No bootstrap required, opened a new log
I20260925 12:24:03.024755 14472 ts_tablet_manager.cc:1397] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20260925 12:24:03.027899 14472 raft_consensus.cc:348] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.028110 14472 raft_consensus.cc:374] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:03.028151 14472 raft_consensus.cc:729] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c051256f32d4440b8bef5bf67b3dfd81, State: Initialized, Role: FOLLOWER
I20260925 12:24:03.028302 14472 consensus_queue.cc:260] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.028667 14171 heartbeater.cc:503] Master 127.1.47.190:35793 was elected leader, sending a full tablet report...
I20260925 12:24:03.028940 14472 ts_tablet_manager.cc:1428] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260925 12:24:03.063572 14172 tablet.cc:2369] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 12:24:03 dist-test-slave-l9ww krb5kdc[12977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339042, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for HTTP/127.1.47.148@KRBTEST.COM
W20260925 12:24:03.212359 14440 tablet.cc:2369] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:24:03.251883 14306 tablet.cc:2369] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:24:03.372581 14478 raft_consensus.cc:482] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:03.372732 14478 raft_consensus.cc:504] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.373167 14478 leader_election.cc:288] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c051256f32d4440b8bef5bf67b3dfd81 (127.1.47.129:37365), 9227960de2e548e4b7b507a503c6b88e (127.1.47.130:45853)
I20260925 12:24:03.379521 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.374109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:46401 (local address 127.1.47.130:45853)
0925 12:24:03.374187 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:03.374188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:03.376056 (+ 1868us) server_negotiation.cc:407] Connection header received
0925 12:24:03.376190 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:03.376191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:03.376205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:03.376236 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:03.376514 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.377249 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.378753 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.379131 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.379220 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.379231 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:03.379409 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 12:24:03.379450 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 12:24:03.379539 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.373713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:45853 (local address 127.1.47.131:46401)
0925 12:24:03.375967 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:03.375981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:03.376091 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:03.376334 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:03.376335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:03.376337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:03.376428 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:03.376430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.377383 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.377384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.378650 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:03.378653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.379246 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.379247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.379324 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.379327 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:03.379391 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:24:03.379438 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2232,"thread_start_us":71,"threads_started":1}
I20260925 12:24:03.380024 14259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1897aadd7e11474a8bbf4f8ec3a17903" candidate_uuid: "681c966183e1482789f82cfa6bc69ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9227960de2e548e4b7b507a503c6b88e" is_pre_election: true
I20260925 12:24:03.380198 14259 raft_consensus.cc:2454] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 681c966183e1482789f82cfa6bc69ef3 in term 0.
I20260925 12:24:03.380486 14328 leader_election.cc:302] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681c966183e1482789f82cfa6bc69ef3, 9227960de2e548e4b7b507a503c6b88e; no voters:
I20260925 12:24:03.380635 14478 raft_consensus.cc:2781] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:03.380694 14478 raft_consensus.cc:482] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:03.380736 14478 raft_consensus.cc:3037] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:03.380851 14469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.374045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:58571 (local address 127.1.47.129:37365)
0925 12:24:03.374153 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:03.374156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:03.376025 (+ 1869us) server_negotiation.cc:407] Connection header received
0925 12:24:03.377499 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:03.377500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:03.377515 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:03.377545 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:03.377775 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.378488 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.379861 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.380425 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.380493 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.380502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:03.380728 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 12:24:03.380773 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 12:24:03.381634 14478 raft_consensus.cc:504] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.381774 14478 leader_election.cc:288] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [CANDIDATE]: Term 1 election: Requested vote from peers c051256f32d4440b8bef5bf67b3dfd81 (127.1.47.129:37365), 9227960de2e548e4b7b507a503c6b88e (127.1.47.130:45853)
I20260925 12:24:03.381970 14259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1897aadd7e11474a8bbf4f8ec3a17903" candidate_uuid: "681c966183e1482789f82cfa6bc69ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9227960de2e548e4b7b507a503c6b88e"
I20260925 12:24:03.382054 14259 raft_consensus.cc:3037] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:03.382705 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.373488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:37365 (local address 127.1.47.131:58571)
0925 12:24:03.375934 (+ 2446us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:03.375949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:03.377384 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:03.377610 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:03.377610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:03.377612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:03.377701 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:03.377703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.378596 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.378596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.379753 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:03.379756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.380533 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.380534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:03.380597 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.380600 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:03.380649 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:24:03.380687 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2422,"thread_start_us":102,"threads_started":1}
I20260925 12:24:03.382771 14259 raft_consensus.cc:2454] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 681c966183e1482789f82cfa6bc69ef3 in term 1.
I20260925 12:24:03.383143 14123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1897aadd7e11474a8bbf4f8ec3a17903" candidate_uuid: "681c966183e1482789f82cfa6bc69ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c051256f32d4440b8bef5bf67b3dfd81" is_pre_election: true
I20260925 12:24:03.383253 14328 leader_election.cc:302] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681c966183e1482789f82cfa6bc69ef3, 9227960de2e548e4b7b507a503c6b88e; no voters:
I20260925 12:24:03.383332 14478 raft_consensus.cc:2781] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:03.383361 14123 raft_consensus.cc:2454] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 681c966183e1482789f82cfa6bc69ef3 in term 0.
I20260925 12:24:03.383150 14124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1897aadd7e11474a8bbf4f8ec3a17903" candidate_uuid: "681c966183e1482789f82cfa6bc69ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c051256f32d4440b8bef5bf67b3dfd81"
I20260925 12:24:03.383481 14478 raft_consensus.cc:686] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 1 LEADER]: Becoming Leader. State: Replica: 681c966183e1482789f82cfa6bc69ef3, State: Running, Role: LEADER
I20260925 12:24:03.383515 14124 raft_consensus.cc:3037] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:03.383569 14478 consensus_queue.cc:237] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } }
I20260925 12:24:03.384483 14124 raft_consensus.cc:2454] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 681c966183e1482789f82cfa6bc69ef3 in term 1.
I20260925 12:24:03.385425 13972 catalog_manager.cc:5495] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 681c966183e1482789f82cfa6bc69ef3 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "681c966183e1482789f82cfa6bc69ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "681c966183e1482789f82cfa6bc69ef3" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 41589 } health_report { overall_health: HEALTHY } } }
I20260925 12:24:03.445480 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.442457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.1.47.131:41589)
0925 12:24:03.442529 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:03.442531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:03.442687 (+ 156us) server_negotiation.cc:407] Connection header received
0925 12:24:03.442775 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:03.442776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:03.442800 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:03.442891 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:03.443002 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.443611 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.444647 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:03.444858 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:03.444939 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:03.445132 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:03.445249 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:03.445341 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:03.445364 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 12:24:03.445389 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 12:24:03.447510 14124 raft_consensus.cc:1261] T 1897aadd7e11474a8bbf4f8ec3a17903 P c051256f32d4440b8bef5bf67b3dfd81 [term 1 FOLLOWER]: Refusing update from remote peer 681c966183e1482789f82cfa6bc69ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:03.447562 14259 raft_consensus.cc:1261] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 1 FOLLOWER]: Refusing update from remote peer 681c966183e1482789f82cfa6bc69ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:03.447908 14478 consensus_queue.cc:1035] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c051256f32d4440b8bef5bf67b3dfd81" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 37365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:03.448076 14489 consensus_queue.cc:1035] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9227960de2e548e4b7b507a503c6b88e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:03.450439 14495 mvcc.cc:205] Tried to move back new op lower bound from 7333228721958043648 to 7333228721699598336. Current Snapshot: MvccSnapshot[applied={T|T < 7333228721958043648}]
I20260925 12:24:03.451412 14496 mvcc.cc:205] Tried to move back new op lower bound from 7333228721958043648 to 7333228721699598336. Current Snapshot: MvccSnapshot[applied={T|T < 7333228721958043648 or (T in {7333228721958043648})}]
I20260925 12:24:03.452665 13973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:03.452775 13973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:03.453578 13973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbf636c588e46e7bae651cbceff45e7: Sending DeleteTablet for 3 replicas of tablet 1897aadd7e11474a8bbf4f8ec3a17903
I20260925 12:24:03.453933 14105 tablet_service.cc:1514] Processing DeleteTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:03 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:57490
I20260925 12:24:03.453935 14239 tablet_service.cc:1514] Processing DeleteTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:03 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:52786
I20260925 12:24:03.453996 14354 tablet_service.cc:1514] Processing DeleteTablet for tablet 1897aadd7e11474a8bbf4f8ec3a17903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:03 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:47512
I20260925 12:24:03.454272 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14012
I20260925 12:24:03.454905 14505 tablet_replica.cc:330] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: stopping tablet replica
I20260925 12:24:03.455009 14505 raft_consensus.cc:2229] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:03.455133 14505 raft_consensus.cc:2258] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:03.455282 14507 tablet_replica.cc:330] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: stopping tablet replica
I20260925 12:24:03.455379 14507 raft_consensus.cc:2229] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:24:03.455504 14507 raft_consensus.cc:2258] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:03.456012 14505 ts_tablet_manager.cc:1905] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:03.458140 14505 ts_tablet_manager.cc:1918] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:03.458274 14505 log.cc:1196] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/1897aadd7e11474a8bbf4f8ec3a17903
I20260925 12:24:03.458287 14507 ts_tablet_manager.cc:1905] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:03.458693 14505 ts_tablet_manager.cc:1939] T 1897aadd7e11474a8bbf4f8ec3a17903 P 9227960de2e548e4b7b507a503c6b88e: Deleting consensus metadata
I20260925 12:24:03.459173 13959 catalog_manager.cc:4841] TS 9227960de2e548e4b7b507a503c6b88e (127.1.47.130:45853): tablet 1897aadd7e11474a8bbf4f8ec3a17903 (table test-table [id=d7765de68f444142b7de106dabbadf13]) successfully deleted
I20260925 12:24:03.460306 14507 ts_tablet_manager.cc:1918] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:03.460394 14507 log.cc:1196] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/1897aadd7e11474a8bbf4f8ec3a17903
I20260925 12:24:03.460654 14507 ts_tablet_manager.cc:1939] T 1897aadd7e11474a8bbf4f8ec3a17903 P 681c966183e1482789f82cfa6bc69ef3: Deleting consensus metadata
I20260925 12:24:03.461061 13960 catalog_manager.cc:4841] TS 681c966183e1482789f82cfa6bc69ef3 (127.1.47.131:41589): tablet 1897aadd7e11474a8bbf4f8ec3a17903 (table test-table [id=d7765de68f444142b7de106dabbadf13]) successfully deleted
W20260925 12:24:03.461612 13957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:37365 (error 108)
I20260925 12:24:03.461961 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14175
I20260925 12:24:03.462154 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:03.462025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:37365 (local address 127.0.0.1:57506)
0925 12:24:03.462070 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:03.462116 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:37365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 12:24:03.462287 13957 catalog_manager.cc:4584] TS c051256f32d4440b8bef5bf67b3dfd81 (127.1.47.129:37365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1897aadd7e11474a8bbf4f8ec3a17903: Network error: Client connection negotiation failed: client connection to 127.1.47.129:37365: connect: Connection refused (error 111)
I20260925 12:24:03.467790 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14308
I20260925 12:24:03.472298 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 13940
I20260925 12:24:03.478438 1214 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 12:24:03.823524 1214 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 12:24:03.823627 1214 mini_ranger.cc:66] Stopping Ranger...
I20260925 12:24:04.168654 1214 mini_ranger.cc:68] Stopped Ranger
2026-09-25 12:24:04.168 UTC [13007] LOG: received smart shutdown request
2026-09-25 12:24:04.173 UTC [13007] LOG: background worker "logical replication launcher" (PID 13016) exited with exit code 1
2026-09-25 12:24:04.173 UTC [13011] LOG: shutting down
2026-09-25 12:24:04.348 UTC [13007] LOG: database system is shut down
2026-09-25T12:24:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50539 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:24:04 dist-test-slave-l9ww krb5kdc[14517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:24:04 dist-test-slave-l9ww krb5kdc[14517](info): set up 2 sockets
Sep 25 12:24:04 dist-test-slave-l9ww krb5kdc[14517](info): commencing operation
krb5kdc: starting...
W20260925 12:24:05.527743 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:24:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:24:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:24:05.650453 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41761
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:05.731531 14530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:05.731747 14530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:05.731770 14530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:05.732980 14530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:05.733024 14530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:05.733039 14530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:05.733052 14530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:05.733064 14530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:05.734494 14530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:05.734687 14530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:05.735158 14530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:05.740420 14535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:05.740477 14538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:05.740420 14536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:05.740610 14530 server_base.cc:1034] running on GCE node
I20260925 12:24:05.740902 14530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:05.741250 14530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:05.742420 14530 hybrid_clock.cc:648] HybridClock initialized: now 1790339045742399 us; error 42 us; skew 500 ppm
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:05.744941 14530 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:05.745326 14530 webserver.cc:460] Webserver started at http://127.1.47.190:36293/ using document root <none> and password file <none>
I20260925 12:24:05.745520 14530 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:05.745560 14530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:05.745646 14530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:05.746644 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "1b4137483236468bac48416a7c6726c1"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "f8bb6b3b2a853c83fb1b3ca012b001cf"
server_key_iv: "fe9b3fe47c800e22b90c339b32537627"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.746933 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "1b4137483236468bac48416a7c6726c1"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "f8bb6b3b2a853c83fb1b3ca012b001cf"
server_key_iv: "fe9b3fe47c800e22b90c339b32537627"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.748251 14530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:24:05.749162 14545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:05.749446 14530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:05.749540 14530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "1b4137483236468bac48416a7c6726c1"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "f8bb6b3b2a853c83fb1b3ca012b001cf"
server_key_iv: "fe9b3fe47c800e22b90c339b32537627"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.749615 14530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:05.764602 14530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:05.766997 14530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:05.767123 14530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:05.771991 14530 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:41761
I20260925 12:24:05.772047 14597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:41761 every 8 connection(s)
I20260925 12:24:05.772441 14530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:05.773376 14598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:05.774966 14598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:05.775537 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14530
I20260925 12:24:05.775674 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:24:05.775964 1214 external_mini_cluster.cc:1428] Setting key d291411100af16a9d131168a389a2be5
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.189@KRBTEST.COM; defaulting to no policy
I20260925 12:24:05.786739 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.775829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.1.47.190:41761)
0925 12:24:05.776063 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:05.776074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:05.776194 (+ 120us) server_negotiation.cc:407] Connection header received
0925 12:24:05.777048 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:05.777050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:05.777068 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:05.777146 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:05.777953 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.779238 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.780246 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.780866 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.781004 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.783084 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:05.783092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:05.783093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:05.783101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:05.785026 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.785501 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.785504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.785505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.785589 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.785830 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.785831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.785831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.786076 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:05.786145 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:05.786379 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 12:24:05.786520 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"spinlock_wait_cycles":10624,"thread_start_us":149,"threads_started":1}
I20260925 12:24:05.786743 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.775820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41761 (local address 127.0.0.1:46958)
0925 12:24:05.776010 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.776068 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:05.776892 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:05.777193 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:05.777196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:05.777632 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:05.777823 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.777828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.779365 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.779367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.780159 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.780162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.780990 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.780991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.781053 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.781609 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:05.781617 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:05.782991 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:05.785137 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.785144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.785146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.785372 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.785657 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.785657 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.785658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.785750 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.786159 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:05.786165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:05.786228 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 12:24:05.786309 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:24:05.786599 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":116,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.189@KRBTEST.COM" created.
I20260925 12:24:05.787595 14598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:05.788007 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.787839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33470)
0925 12:24:05.787914 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.787969 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 12:24:05.788161 14548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
I20260925 12:24:05.788667 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.788549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33486)
0925 12:24:05.788600 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.788634 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 12:24:05.788828 14598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:44897: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
I20260925 12:24:05.803603 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:44897
--webserver_interface=127.1.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:05.838099 14598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } attempt: 1
I20260925 12:24:05.838692 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.838538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33492)
0925 12:24:05.838607 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.838655 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 12:24:05.839064 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.838973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33500)
0925 12:24:05.839017 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.839045 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 12:24:05.839232 14598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:44897: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
W20260925 12:24:05.886765 14605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:05.886991 14605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:05.887015 14605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:05.888290 14605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:05.888336 14605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:05.888350 14605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:05.888363 14605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:05.888375 14605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:05.889914 14605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:05.890187 14605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:05.890434 14605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:05.895457 14611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:05.895463 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:05.895484 14613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:05.895727 14605 server_base.cc:1034] running on GCE node
I20260925 12:24:05.896015 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:05.896337 14605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:05.897544 14605 hybrid_clock.cc:648] HybridClock initialized: now 1790339045897531 us; error 47 us; skew 500 ppm
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:05.900341 14605 init.cc:377] Logged in from keytab as kudu/127.1.47.189@KRBTEST.COM (short username kudu)
I20260925 12:24:05.900833 14605 webserver.cc:460] Webserver started at http://127.1.47.189:46265/ using document root <none> and password file <none>
I20260925 12:24:05.901086 14605 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:05.901132 14605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:05.901289 14605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:05.902369 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data/instance:
uuid: "f592ad3d3b1c42c5b2102b92533ed538"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "44a06d432b5dfbb23d2bf2ff9659403c"
server_key_iv: "03da79b289e893a72fbe33b6bbd7cf69"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.902725 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal/instance:
uuid: "f592ad3d3b1c42c5b2102b92533ed538"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "44a06d432b5dfbb23d2bf2ff9659403c"
server_key_iv: "03da79b289e893a72fbe33b6bbd7cf69"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.904175 14605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 12:24:05.904980 14620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:05.905172 14605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:05.905268 14605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "f592ad3d3b1c42c5b2102b92533ed538"
format_stamp: "Formatted at 2026-09-25 12:24:05 on dist-test-slave-l9ww"
server_key: "44a06d432b5dfbb23d2bf2ff9659403c"
server_key_iv: "03da79b289e893a72fbe33b6bbd7cf69"
server_key_version: "encryptionkey@0"
I20260925 12:24:05.905325 14605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:05.907478 14598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } attempt: 2
I20260925 12:24:05.908030 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.907894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33512)
0925 12:24:05.907951 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.908001 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 12:24:05.908401 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.908326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33522)
0925 12:24:05.908355 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.908382 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 12:24:05.908519 14598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:44897: Network error: Client connection negotiation failed: client connection to 127.1.47.189:44897: connect: Connection refused (error 111)
I20260925 12:24:05.918579 14605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:05.921018 14605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:05.921132 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:05.924995 14605 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.189:44897
I20260925 12:24:05.925052 14672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.189:44897 every 8 connection(s)
I20260925 12:24:05.925375 14605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
I20260925 12:24:05.926261 14673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:05.927816 14673 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:05.928751 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14605
I20260925 12:24:05.928850 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-1/wal/instance
I20260925 12:24:05.929047 1214 external_mini_cluster.cc:1428] Setting key 6e8a47690177d1981701d8d5bc736a16
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.188@KRBTEST.COM; defaulting to no policy
I20260925 12:24:05.938292 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.928649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.1.47.190:41761)
0925 12:24:05.928713 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:05.928715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:05.928942 (+ 227us) server_negotiation.cc:407] Connection header received
0925 12:24:05.929484 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:05.929486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:05.929503 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:05.929565 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:05.930299 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.930789 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.932430 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.932700 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.932761 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.935257 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:05.935260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:05.935260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:05.935272 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:05.936830 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.937246 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.937248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.937248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.937308 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.937593 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.937594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.937595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.937713 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:05.937794 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:05.938079 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 12:24:05.938162 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 12:24:05.938432 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.928618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41761 (local address 127.0.0.1:46960)
0925 12:24:05.928819 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.928866 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:05.929375 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:05.929637 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:05.929641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:05.929893 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:05.930193 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.930200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.930885 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.930886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.932330 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.932334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.932797 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.932799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.932971 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.933484 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:05.933497 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:05.935128 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:05.936925 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.936932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.936934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.937168 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.937406 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.937407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.937407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.937522 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.937784 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:05.937791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:05.937933 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 12:24:05.938045 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 12:24:05.938203 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.188@KRBTEST.COM" created.
I20260925 12:24:05.939007 14673 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:24:05.947608 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.939292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33532)
0925 12:24:05.939343 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.939354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:05.939396 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:05.939685 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:05.939686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:05.939847 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:05.939929 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.939931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.941097 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.941098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.941590 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:05.941591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.942257 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.942258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:05.942312 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.942786 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:05.942791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:05.944146 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:05.946412 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.946413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.946413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.946585 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.946831 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:05.946832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:05.946832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:05.946900 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:05.947262 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:05.947262 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:05.947301 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 12:24:05.947397 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 12:24:05.947490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:24:05.947629 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.939377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.1.47.189:44897)
0925 12:24:05.939538 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:05.939542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:05.939559 (+ 17us) server_negotiation.cc:407] Connection header received
0925 12:24:05.939586 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:05.939588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:05.939598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:05.939652 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:05.940001 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.941006 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.941672 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:05.942178 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:05.942255 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:05.944237 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:05.944248 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:05.944249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:05.944257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:05.946298 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.946674 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.946678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.946678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.946742 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:05.946987 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:05.946988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:05.946988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:05.947185 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:05.947260 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:05.947423 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 12:24:05.947541 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260925 12:24:05.948493 14673 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:05.948870 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.948761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38664)
0925 12:24:05.948797 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.948851 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260925 12:24:05.949065 14622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
I20260925 12:24:05.949589 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.949474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38674)
0925 12:24:05.949525 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.949558 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:24:05.949709 14673 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:43297: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
I20260925 12:24:05.955127 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:43297
--webserver_interface=127.1.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:05.998983 14673 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } attempt: 1
I20260925 12:24:05.999578 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.999438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38682)
0925 12:24:05.999507 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.999547 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 12:24:05.999658 14598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } attempt: 3
I20260925 12:24:05.999989 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.999893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38688)
0925 12:24:05.999932 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:05.999966 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260925 12:24:06.000083 14673 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:43297: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Sep 25 12:24:05 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:24:06.006886 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.000008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33544)
0925 12:24:06.000059 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.000069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.000123 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.000274 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.000274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.000439 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.000501 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.000502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.001186 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.001187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.001704 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.001706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.002090 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.002091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.002173 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.002644 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.002652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.003814 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.005690 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.005691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.005692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.005899 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.006142 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.006142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.006143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.006214 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.006512 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.006513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.006564 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.006652 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.006766 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 12:24:06.006886 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:05.999997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.1.47.189:44897)
0925 12:24:06.000031 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.000034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.000140 (+ 106us) server_negotiation.cc:407] Connection header received
0925 12:24:06.000173 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.000175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.000187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.000268 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.000583 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.001088 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.001801 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.002015 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.002094 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.003984 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.003985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.003985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.003992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.005561 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.005998 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.005999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.006000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.006065 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.006287 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.006288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.006289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.006427 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.006509 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.006694 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.006793 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:24:06.007292 14598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:06.007694 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.007584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38692)
0925 12:24:06.007643 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.007674 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 12:24:06.008039 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.007945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38702)
0925 12:24:06.007994 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.008019 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 12:24:06.008149 14598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:43297: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
W20260925 12:24:06.037968 14680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:06.038223 14680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:06.038261 14680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 12:24:06.039312 14598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } attempt: 1
W20260925 12:24:06.039531 14680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:06.039603 14680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:06.039630 14680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:06.039660 14680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:06.039690 14680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:06.039810 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.039691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38716)
0925 12:24:06.039738 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.039781 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 12:24:06.040210 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.040108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38720)
0925 12:24:06.040159 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.040186 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 12:24:06.040302 14598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:43297: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
I20260925 12:24:06.041553 14680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41761,127.1.47.189:44897,127.1.47.188:43297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:43297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:06.041848 14680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:06.042088 14680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:06.046934 14686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:06.046926 14688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:06.046934 14685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:06.047099 14680 server_base.cc:1034] running on GCE node
I20260925 12:24:06.047390 14680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:06.047688 14680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:06.048838 14680 hybrid_clock.cc:648] HybridClock initialized: now 1790339046048814 us; error 39 us; skew 500 ppm
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:06.051589 14680 init.cc:377] Logged in from keytab as kudu/127.1.47.188@KRBTEST.COM (short username kudu)
I20260925 12:24:06.052165 14680 webserver.cc:460] Webserver started at http://127.1.47.188:38515/ using document root <none> and password file <none>
I20260925 12:24:06.052490 14680 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:06.052564 14680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:06.052702 14680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:06.053891 14680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data/instance:
uuid: "af65aebf25b342d5a1207afe53cae873"
format_stamp: "Formatted at 2026-09-25 12:24:06 on dist-test-slave-l9ww"
server_key: "dab0643f12b7125629a534ab38e5508a"
server_key_iv: "fa96b1210940de63c6dab01b639bf416"
server_key_version: "encryptionkey@0"
I20260925 12:24:06.054261 14680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal/instance:
uuid: "af65aebf25b342d5a1207afe53cae873"
format_stamp: "Formatted at 2026-09-25 12:24:06 on dist-test-slave-l9ww"
server_key: "dab0643f12b7125629a534ab38e5508a"
server_key_iv: "fa96b1210940de63c6dab01b639bf416"
server_key_version: "encryptionkey@0"
I20260925 12:24:06.055636 14680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:24:06.056485 14695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:06.056670 14680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:06.056768 14680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "af65aebf25b342d5a1207afe53cae873"
format_stamp: "Formatted at 2026-09-25 12:24:06 on dist-test-slave-l9ww"
server_key: "dab0643f12b7125629a534ab38e5508a"
server_key_iv: "fa96b1210940de63c6dab01b639bf416"
server_key_version: "encryptionkey@0"
I20260925 12:24:06.056859 14680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:06.068233 14673 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } attempt: 2
I20260925 12:24:06.068650 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.068522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38726)
0925 12:24:06.068585 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.068622 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 12:24:06.068971 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.068878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38728)
0925 12:24:06.068924 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.068948 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:24:06.069063 14673 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:43297: Network error: Client connection negotiation failed: client connection to 127.1.47.188:43297: connect: Connection refused (error 111)
I20260925 12:24:06.069403 14680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:06.071290 14680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:06.071455 14680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:06.075508 14680 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.188:43297
I20260925 12:24:06.075542 14747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.188:43297 every 8 connection(s)
I20260925 12:24:06.075953 14680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
I20260925 12:24:06.076869 14748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:06.078428 14748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:06.080329 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14680
I20260925 12:24:06.080440 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790338888974512-1214-0/minicluster-data/master-2/wal/instance
I20260925 12:24:06.080655 1214 external_mini_cluster.cc:1428] Setting key f09a4e15389d387c038f1e8112cf7aa0
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:24:06.089447 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.079206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.1.47.190:41761)
0925 12:24:06.079269 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.079271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.079484 (+ 213us) server_negotiation.cc:407] Connection header received
0925 12:24:06.079998 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.080000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.080010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.080077 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.081021 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.081730 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.082738 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.082973 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.083064 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.085474 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.085476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.085476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.085485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.087760 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.088270 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.088271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.088271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.088333 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.088666 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.088666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.088667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.088837 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.088911 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.089260 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.089348 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:24:06.089632 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.079161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41761 (local address 127.0.0.1:46968)
0925 12:24:06.079343 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.079397 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.079837 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.080158 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.080163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.080615 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.080908 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.080913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.081834 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.081835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.082642 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.082644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.083072 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.083073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.083204 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.083762 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.083772 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.085380 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.087842 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.087849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.087851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.088192 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.088451 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.088452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.088452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.088587 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.088929 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.088937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.089112 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.089191 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.089361 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260925 12:24:06.089649 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.081029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.1.47.188:43297)
0925 12:24:06.081386 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.081391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.081413 (+ 22us) server_negotiation.cc:407] Connection header received
0925 12:24:06.081433 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.081435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.081454 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.081524 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.082114 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.082807 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.083614 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.084085 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.084159 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.086540 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.086547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.086547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.086553 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.088340 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.088718 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.088721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.088721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.088773 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.089056 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.089056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.089057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.089211 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.089416 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.089465 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.089550 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":92,"threads_started":1}
I20260925 12:24:06.090258 14748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:06.098492 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.090598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.1.47.189:44897)
0925 12:24:06.090671 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.090673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.090706 (+ 33us) server_negotiation.cc:407] Connection header received
0925 12:24:06.090775 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.090776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.090783 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.090824 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.091136 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.091968 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.093014 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.093303 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.093377 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.095474 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.095475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.095476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.095485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.097226 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.097569 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.097570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.097570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.097632 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.097916 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.097917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.097917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.098029 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.098080 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.098297 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.098389 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:24:06.098515 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.090513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:44897 (local address 127.0.0.1:33560)
0925 12:24:06.090643 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.090672 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.090717 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.090847 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.090848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.090994 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.091064 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.091066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.092104 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.092106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.092887 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.092889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.093384 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.093385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.093465 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.094008 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.094013 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.095382 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.097303 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.097305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.097305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.097471 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.097732 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.097733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.097733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.097798 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.098098 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.098099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.098156 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.098231 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.098349 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260925 12:24:06.099057 14748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:24:06.099140 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.090482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.1.47.190:41761)
0925 12:24:06.090576 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.090578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.090585 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:24:06.090604 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.090604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.090613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.090674 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.091290 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.092024 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.092997 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.093289 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.093366 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.096313 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.096314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.096314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.096322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.097789 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.098183 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.098184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.098184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.098257 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.098480 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.098481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.098481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.098623 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.098826 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.098979 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.099066 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:24:06.107093 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.099360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38736)
0925 12:24:06.099401 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.099413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.099460 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.099582 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.099583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.099732 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.099769 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.099771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.100739 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.100740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.101634 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.101637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.102155 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.102156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.102206 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.102772 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.102778 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.104443 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.106084 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.106085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.106086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.106246 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.106448 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.106449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.106449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.106504 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.106743 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.106744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.106781 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.106865 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.106973 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 12:24:06.107093 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.099434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.1.47.188:43297)
0925 12:24:06.099474 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.099475 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.099483 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:06.099505 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.099506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.099519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.099573 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.099875 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.100637 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.101732 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.102051 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.102126 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.104531 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.104532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.104532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.104539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.105986 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.106330 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.106330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.106331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.106384 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.106565 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.106566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.106567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.106671 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.106741 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.106887 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.107016 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":21}
I20260925 12:24:06.108585 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.099903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.1.47.189:44897)
0925 12:24:06.099940 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.099942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.099949 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:24:06.099968 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.099969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.099977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.100026 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.100620 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.101495 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.102778 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.103100 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.103185 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.105818 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.105820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.105820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.105827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.107331 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.107670 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.107672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.107672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.107754 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.107991 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.107992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.107992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.108135 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.108334 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.108415 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.108499 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:24:06.108933 14748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: Bootstrap starting.
I20260925 12:24:06.110064 14748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:06.110106 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:24:06.110373 14748 log.cc:824] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:06.111280 14748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: No bootstrap required, opened a new log
I20260925 12:24:06.113260 14748 raft_consensus.cc:348] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.113421 14748 raft_consensus.cc:374] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:06.113473 14748 raft_consensus.cc:729] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af65aebf25b342d5a1207afe53cae873, State: Initialized, Role: FOLLOWER
I20260925 12:24:06.113598 14748 consensus_queue.cc:260] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.113899 14748 sys_catalog.cc:564] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:06.113879 14761 sys_catalog.cc:455] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.113991 14761 sys_catalog.cc:458] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:24:06.115978 14598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } attempt: 2
W20260925 12:24:06.116852 14772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:24:06.120290 14772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:24:06.118448 14680 master_runner.cc:186] Error getting master registration for 127.1.47.190:41761: OK, instance_id { permanent_uuid: "1b4137483236468bac48416a7c6726c1" instance_seqno: 1790339045770716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:24:06.121726 14680 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 12:24:06.122750 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.111715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33590 (local address 127.1.47.189:44897)
0925 12:24:06.111756 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.111758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.111945 (+ 187us) server_negotiation.cc:407] Connection header received
0925 12:24:06.112022 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.112023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.112035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.112099 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.112917 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.113438 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.114454 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.114707 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.114807 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.117831 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.117833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.117833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.117842 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.120088 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.120570 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.120572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.120572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.120635 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.121129 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.121131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.121131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.121283 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.121710 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.122550 (+ 840us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.122644 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:24:06.123541 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.112034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.1.47.188:43297)
0925 12:24:06.112076 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.112078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.115489 (+ 3411us) server_negotiation.cc:407] Connection header received
0925 12:24:06.115575 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.115577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.115588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.115661 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.116438 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.117073 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.118534 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.118809 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.118846 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.120580 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.120581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.120581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.120587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.122223 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.122650 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.122651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.122652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.122703 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.122905 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.122906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.122906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.123062 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.123140 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.123344 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.123418 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:24:06.124420 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.111794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.1.47.190:41761)
0925 12:24:06.111871 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.111874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.112183 (+ 309us) server_negotiation.cc:407] Connection header received
0925 12:24:06.112231 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.112232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.112242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.112292 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.112948 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.113880 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.116327 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.116600 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.116660 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.118095 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.118096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.118096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.118103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.123037 (+ 4934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.123406 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.123407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.123407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.123461 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.123659 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.123660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.123660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.123790 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.123897 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.124246 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.124321 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:24:06.132212 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.116247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38756)
0925 12:24:06.117502 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.117513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.117602 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.119314 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.119315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.119490 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.119534 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.119536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.124373 (+ 4837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.124374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.125251 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.125253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.125637 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.125639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.125705 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.126472 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.126478 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.128014 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.131099 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.131101 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.131101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.131288 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.131576 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.131576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.131577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.131636 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.131966 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.131967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.132008 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.132064 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.132145 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235}
I20260925 12:24:06.132337 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.116351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.1.47.188:43297)
0925 12:24:06.119136 (+ 2785us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.119138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.119145 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:24:06.119164 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.119165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.119174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.120841 (+ 1667us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.120895 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.124291 (+ 3396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.125340 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.125550 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.125603 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.128107 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.128108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.128108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.128116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.130985 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.131416 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.131417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.131418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.131486 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.131713 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.131714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.131715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.131898 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.131975 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.132116 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.132207 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2764,"thread_start_us":65,"threads_started":1}
I20260925 12:24:06.133982 14598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: Bootstrap starting.
I20260925 12:24:06.134768 14598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:06.135100 14598 log.cc:824] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:06.136056 14598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: No bootstrap required, opened a new log
I20260925 12:24:06.138311 14598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.138479 14598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:06.138518 14598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4137483236468bac48416a7c6726c1, State: Initialized, Role: FOLLOWER
I20260925 12:24:06.138654 14598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.138991 14598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:06.138957 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.139067 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:24:06.141146 14787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:24:06.141259 14787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:24:06.141703 14530 master_runner.cc:186] Error getting master registration for 127.1.47.189:44897: OK, instance_id { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" instance_seqno: 1790339045923774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:24:06.143743 14530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 12:24:06.160193 14673 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } attempt: 3
Sep 25 12:24:06 dist-test-slave-l9ww krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:24:06.167550 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.160584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.1.47.188:43297)
0925 12:24:06.160640 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:06.160642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:06.160657 (+ 15us) server_negotiation.cc:407] Connection header received
0925 12:24:06.160694 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:06.160695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:06.160707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:06.160793 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:06.161133 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.161639 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.162379 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.162607 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.162687 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.164922 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:06.164924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:06.164924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:06.164932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:06.166379 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.166755 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.166757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.166757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.166816 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:06.167017 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:06.167018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:06.167018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:06.167135 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:06.167210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:06.167374 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:24:06.167454 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:24:06.167550 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:06.160535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:43297 (local address 127.0.0.1:38760)
0925 12:24:06.160592 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:06.160605 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:06.160652 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:06.160823 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:06.160824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:06.161000 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:06.161045 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.161047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.161730 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.161731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.162293 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:06.162295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:06.162683 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:06.162685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:06.162740 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:06.163524 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:06.163529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:06.164834 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:06.166511 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.166512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.166512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.166682 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.166884 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:06.166885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:06.166885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:06.166949 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:06.167202 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:06.167203 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:06.167250 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 12:24:06.167334 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 12:24:06.167440 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 12:24:06.169281 14673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: Bootstrap starting.
I20260925 12:24:06.169946 14673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:06.170253 14673 log.cc:824] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:06.171036 14673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: No bootstrap required, opened a new log
I20260925 12:24:06.172688 14673 raft_consensus.cc:348] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.172833 14673 raft_consensus.cc:374] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:06.172880 14673 raft_consensus.cc:729] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f592ad3d3b1c42c5b2102b92533ed538, State: Initialized, Role: FOLLOWER
I20260925 12:24:06.173025 14673 consensus_queue.cc:260] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.173810 14790 sys_catalog.cc:455] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.173879 14673 sys_catalog.cc:564] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:06.173954 14790 sys_catalog.cc:458] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:24:06.175740 14801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:24:06.175836 14801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:24:06.178010 14605 master_runner.cc:263] No leader master found from master f592ad3d3b1c42c5b2102b92533ed538
I20260925 12:24:06.178052 14605 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 12:24:06.268008 14551 tablet.cc:2369] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:24:06.290084 14776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:06.290215 14776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.290462 14776 leader_election.cc:288] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f592ad3d3b1c42c5b2102b92533ed538 (127.1.47.189:44897), af65aebf25b342d5a1207afe53cae873 (127.1.47.188:43297)
I20260925 12:24:06.291113 14647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b4137483236468bac48416a7c6726c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f592ad3d3b1c42c5b2102b92533ed538" is_pre_election: true
I20260925 12:24:06.291271 14647 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4137483236468bac48416a7c6726c1 in term 0.
I20260925 12:24:06.291555 14548 leader_election.cc:302] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4137483236468bac48416a7c6726c1, f592ad3d3b1c42c5b2102b92533ed538; no voters:
I20260925 12:24:06.291832 14722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b4137483236468bac48416a7c6726c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af65aebf25b342d5a1207afe53cae873" is_pre_election: true
I20260925 12:24:06.292021 14722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4137483236468bac48416a7c6726c1 in term 0.
I20260925 12:24:06.292435 14776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:06.292492 14776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:06.292513 14776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:06.293459 14776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.293589 14776 leader_election.cc:288] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [CANDIDATE]: Term 1 election: Requested vote from peers f592ad3d3b1c42c5b2102b92533ed538 (127.1.47.189:44897), af65aebf25b342d5a1207afe53cae873 (127.1.47.188:43297)
I20260925 12:24:06.293954 14647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b4137483236468bac48416a7c6726c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f592ad3d3b1c42c5b2102b92533ed538"
I20260925 12:24:06.294044 14647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:06.294921 14647 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4137483236468bac48416a7c6726c1 in term 1.
I20260925 12:24:06.295153 14548 leader_election.cc:302] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4137483236468bac48416a7c6726c1, f592ad3d3b1c42c5b2102b92533ed538; no voters:
I20260925 12:24:06.295313 14722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b4137483236468bac48416a7c6726c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af65aebf25b342d5a1207afe53cae873"
I20260925 12:24:06.295393 14722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:06.296304 14722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4137483236468bac48416a7c6726c1 in term 1.
I20260925 12:24:06.296546 14776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:06.296660 14776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1b4137483236468bac48416a7c6726c1, State: Running, Role: LEADER
I20260925 12:24:06.296741 14776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } }
I20260925 12:24:06.297119 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4137483236468bac48416a7c6726c1. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.297179 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:06.297483 14805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:06.297689 14805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:06.300922 14722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [term 1 FOLLOWER]: Refusing update from remote peer 1b4137483236468bac48416a7c6726c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:06.301188 14647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [term 1 FOLLOWER]: Refusing update from remote peer 1b4137483236468bac48416a7c6726c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:06.301744 14776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:06.301911 14776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:06.303331 14790 sys_catalog.cc:455] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4137483236468bac48416a7c6726c1. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.303422 14790 sys_catalog.cc:458] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:24:06.304483 14761 sys_catalog.cc:455] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4137483236468bac48416a7c6726c1. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.304570 14761 sys_catalog.cc:458] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:24:06.306399 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.306488 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:06.306944 14761 sys_catalog.cc:455] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.307032 14761 sys_catalog.cc:458] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:24:06.307260 14806 mvcc.cc:205] Tried to move back new op lower bound from 7333228733645705216 to 7333228733631823872. Current Snapshot: MvccSnapshot[applied={T|T < 7333228733645705216}]
I20260925 12:24:06.307446 14809 mvcc.cc:205] Tried to move back new op lower bound from 7333228733645705216 to 7333228733631823872. Current Snapshot: MvccSnapshot[applied={T|T < 7333228733645705216}]
I20260925 12:24:06.307876 14776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.307948 14776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:06.308315 14790 sys_catalog.cc:455] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4137483236468bac48416a7c6726c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4137483236468bac48416a7c6726c1" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41761 } } peers { permanent_uuid: "f592ad3d3b1c42c5b2102b92533ed538" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 44897 } } peers { permanent_uuid: "af65aebf25b342d5a1207afe53cae873" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 43297 } } }
I20260925 12:24:06.308409 14790 sys_catalog.cc:458] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:24:06.309116 14807 mvcc.cc:205] Tried to move back new op lower bound from 7333228733645705216 to 7333228733631823872. Current Snapshot: MvccSnapshot[applied={T|T < 7333228733645705216}]
I20260925 12:24:06.310187 14805 catalog_manager.cc:1347] Generated new cluster ID: 7acc1ea7ff0241938fe2c788fe426190
I20260925 12:24:06.310253 14805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:06.329180 14805 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:24:06.329902 14805 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:06.338020 14805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b4137483236468bac48416a7c6726c1: Generated new TSK 0
I20260925 12:24:06.338243 14805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:24:07.120893 14772 catalog_manager.cc:1259] Loaded cluster ID: 7acc1ea7ff0241938fe2c788fe426190
I20260925 12:24:07.120980 14772 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: loading cluster ID for follower catalog manager: success
I20260925 12:24:07.121935 14772 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: acquiring CA information for follower catalog manager: success
I20260925 12:24:07.122421 14772 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af65aebf25b342d5a1207afe53cae873: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:24:07.176318 14801 catalog_manager.cc:1259] Loaded cluster ID: 7acc1ea7ff0241938fe2c788fe426190
I20260925 12:24:07.176394 14801 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: loading cluster ID for follower catalog manager: success
I20260925 12:24:07.177266 14801 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: acquiring CA information for follower catalog manager: success
I20260925 12:24:07.177604 14801 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f592ad3d3b1c42c5b2102b92533ed538: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:24:07.391494 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14530
I20260925 12:24:07.397693 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14605
I20260925 12:24:07.402614 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14680
2026-09-25T12:24:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2928 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:24:07 dist-test-slave-l9ww krb5kdc[14822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:24:07 dist-test-slave-l9ww krb5kdc[14822](info): set up 2 sockets
Sep 25 12:24:07 dist-test-slave-l9ww krb5kdc[14822](info): commencing operation
krb5kdc: starting...
W20260925 12:24:08.444028 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:24:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:24:08.563446 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:34755
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:08.645036 14835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:08.645290 14835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:08.645329 14835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:08.646512 14835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:08.646579 14835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:08.646606 14835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:08.646636 14835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:08.646659 14835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:08.648286 14835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:08.648566 14835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:08.648789 14835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:08.653781 14841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:08.653786 14843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:08.653733 14840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:08.654052 14835 server_base.cc:1034] running on GCE node
I20260925 12:24:08.654299 14835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:08.654639 14835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:08.655915 14835 hybrid_clock.cc:648] HybridClock initialized: now 1790339048655830 us; error 37 us; skew 500 ppm
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:08.658936 14835 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:08.659484 14835 webserver.cc:460] Webserver started at http://127.1.47.190:42249/ using document root <none> and password file <none>
I20260925 12:24:08.659741 14835 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:08.659826 14835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:08.660076 14835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:08.661276 14835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "07c478d720f6429382d98d01d1e84282"
format_stamp: "Formatted at 2026-09-25 12:24:08 on dist-test-slave-l9ww"
server_key: "e3b3861d268a7b83bcf7a91a3cf6961a"
server_key_iv: "e6b43b63e97144505395eb945371f8ea"
server_key_version: "encryptionkey@0"
I20260925 12:24:08.661674 14835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "07c478d720f6429382d98d01d1e84282"
format_stamp: "Formatted at 2026-09-25 12:24:08 on dist-test-slave-l9ww"
server_key: "e3b3861d268a7b83bcf7a91a3cf6961a"
server_key_iv: "e6b43b63e97144505395eb945371f8ea"
server_key_version: "encryptionkey@0"
I20260925 12:24:08.663153 14835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:24:08.664002 14850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:08.664177 14835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:08.664281 14835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "07c478d720f6429382d98d01d1e84282"
format_stamp: "Formatted at 2026-09-25 12:24:08 on dist-test-slave-l9ww"
server_key: "e3b3861d268a7b83bcf7a91a3cf6961a"
server_key_iv: "e6b43b63e97144505395eb945371f8ea"
server_key_version: "encryptionkey@0"
I20260925 12:24:08.664367 14835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:08.674450 14835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:08.676374 14835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:08.676542 14835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:08.680308 14835 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:34755
I20260925 12:24:08.680341 14902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:34755 every 8 connection(s)
I20260925 12:24:08.680703 14835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:08.681596 14903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:08.683940 14903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282: Bootstrap starting.
I20260925 12:24:08.684518 14903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:08.684816 14903 log.cc:824] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:08.685499 14903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282: No bootstrap required, opened a new log
I20260925 12:24:08.687075 14903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } }
I20260925 12:24:08.687214 14903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:08.687237 14903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c478d720f6429382d98d01d1e84282, State: Initialized, Role: FOLLOWER
I20260925 12:24:08.687391 14903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } }
I20260925 12:24:08.687456 14903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:08.687477 14903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:08.687525 14903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:08.688126 14903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } }
I20260925 12:24:08.688259 14903 leader_election.cc:302] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07c478d720f6429382d98d01d1e84282; no voters:
I20260925 12:24:08.688369 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14835
I20260925 12:24:08.688439 14903 leader_election.cc:288] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:24:08.688475 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:24:08.688514 14906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:08.688668 14906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [term 1 LEADER]: Becoming Leader. State: Replica: 07c478d720f6429382d98d01d1e84282, State: Running, Role: LEADER
I20260925 12:24:08.688678 1214 external_mini_cluster.cc:1428] Setting key c999ac370ca051a996dd833016dcbc30
I20260925 12:24:08.688776 14903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:08.688804 14906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } }
I20260925 12:24:08.689436 14907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07c478d720f6429382d98d01d1e84282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } } }
I20260925 12:24:08.689520 14907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:08.689865 14917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:08.689881 14908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07c478d720f6429382d98d01d1e84282. Latest consensus state: current_term: 1 leader_uuid: "07c478d720f6429382d98d01d1e84282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07c478d720f6429382d98d01d1e84282" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 34755 } } }
I20260925 12:24:08.690039 14908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:08.690424 14917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:08.692219 14917 catalog_manager.cc:1347] Generated new cluster ID: 48b9c92bae3d4213af936e97391652a0
I20260925 12:24:08.692274 14917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:08.699759 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:08.689321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.1.47.190:34755)
0925 12:24:08.689489 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:08.689500 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:08.689756 (+ 256us) server_negotiation.cc:407] Connection header received
0925 12:24:08.689891 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:08.689900 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:08.689969 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:08.690132 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:08.691064 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:08.692339 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:08.693133 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:08.693583 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:08.693704 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:08.696240 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:08.696244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:08.696245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:08.696264 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:08.698081 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:08.698469 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:08.698478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:08.698479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:08.698593 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:08.698809 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:08.698810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:08.698811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:08.699075 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:08.699254 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:08.699325 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 12:24:08.699575 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260925 12:24:08.700763 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:24:08.716408 14917 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:24:08.717005 14917 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:08.722417 14917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07c478d720f6429382d98d01d1e84282: Generated new TSK 0
I20260925 12:24:08.722638 14917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:24:08.754820 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14835
2026-09-25T12:24:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1350 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14930](info): set up 2 sockets
Sep 25 12:24:08 dist-test-slave-l9ww krb5kdc[14930](info): commencing operation
krb5kdc: starting...
W20260925 12:24:09.884059 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:24:09 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:24:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:24:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:24:10.006512 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:10.087335 14943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:10.087555 14943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:10.087579 14943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:10.088781 14943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:10.088826 14943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:10.088841 14943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:10.088853 14943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:10.088898 14943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:10.090363 14943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:10.090623 14943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:10.090920 14943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:10.096105 14949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.096274 14943 server_base.cc:1034] running on GCE node
W20260925 12:24:10.096091 14948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.096086 14951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.096671 14943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:10.097019 14943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:10.098212 14943 hybrid_clock.cc:648] HybridClock initialized: now 1790339050098184 us; error 37 us; skew 500 ppm
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:10.100981 14943 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:10.101577 14943 webserver.cc:460] Webserver started at http://127.1.47.190:41521/ using document root <none> and password file <none>
I20260925 12:24:10.101833 14943 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:10.101908 14943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:10.102067 14943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:10.103255 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "0abca66feadb45bea1ba04e67154adb0"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "65ba65e4beb4a0a89c1ad3fe6e3519ae"
server_key_iv: "641306946145e32435ec6b8aa772abd9"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.103669 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "0abca66feadb45bea1ba04e67154adb0"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "65ba65e4beb4a0a89c1ad3fe6e3519ae"
server_key_iv: "641306946145e32435ec6b8aa772abd9"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.105270 14943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:10.106110 14958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.106288 14943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:10.106400 14943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "0abca66feadb45bea1ba04e67154adb0"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "65ba65e4beb4a0a89c1ad3fe6e3519ae"
server_key_iv: "641306946145e32435ec6b8aa772abd9"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.106499 14943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:10.113104 14943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:10.115373 14943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:10.115553 14943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:10.119658 14943 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37349
I20260925 12:24:10.119696 15010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37349 every 8 connection(s)
I20260925 12:24:10.120107 14943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:10.121279 15011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:10.121462 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 14943
I20260925 12:24:10.121570 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:24:10.121789 1214 external_mini_cluster.cc:1428] Setting key 4f904fce949e8a82b630f9d4441f3384
I20260925 12:24:10.125277 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap starting.
I20260925 12:24:10.126148 15011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:10.126497 15011 log.cc:824] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:10.127317 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: No bootstrap required, opened a new log
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:10.130028 15011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.130167 15011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:10.130189 15011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Initialized, Role: FOLLOWER
I20260925 12:24:10.130332 15011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.130405 15011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:10.130427 15011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:10.130484 15011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:10.131039 15011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.131125 15011 leader_election.cc:302] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0abca66feadb45bea1ba04e67154adb0; no voters:
I20260925 12:24:10.131332 15011 leader_election.cc:288] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:24:10.131362 15016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:10.131477 15016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Running, Role: LEADER
I20260925 12:24:10.131636 15011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:10.131620 15016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.132300 15018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0abca66feadb45bea1ba04e67154adb0. Latest consensus state: current_term: 1 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:10.132388 15018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:10.132678 15025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:10.133237 15025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:10.134128 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:10.134239 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:10.134444 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.122480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.1.47.190:37349)
0925 12:24:10.122655 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.122672 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.122713 (+ 41us) server_negotiation.cc:407] Connection header received
0925 12:24:10.123214 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.123222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.123275 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.123425 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:10.124451 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.125519 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.126783 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.127324 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.127462 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.130351 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:10.130355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:10.130356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:10.130375 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:10.132521 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.132958 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.132961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.132962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.133063 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.133253 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.133253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.133253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.133552 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:10.133723 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.134046 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.134199 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260925 12:24:10.135057 15025 catalog_manager.cc:1347] Generated new cluster ID: 5cfcd29934de46fea103f9bfbf349785
I20260925 12:24:10.135109 15025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:10.160840 15025 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:24:10.161469 15025 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:10.166437 15025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Generated new TSK 0
I20260925 12:24:10.166668 15025 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:24:10.200057 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:10.305640 15037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:10.305847 15037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:10.305871 15037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:10.306957 15037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:10.306991 15037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:10.307016 15037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:10.308391 15037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:10.308620 15037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:10.308831 15037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:10.314023 15043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.314085 15037 server_base.cc:1034] running on GCE node
W20260925 12:24:10.314034 15045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.314029 15042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.314448 15037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:10.314702 15037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:10.315896 15037 hybrid_clock.cc:648] HybridClock initialized: now 1790339050315872 us; error 39 us; skew 500 ppm
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:10.318432 15037 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:10.318830 15037 webserver.cc:460] Webserver started at http://127.1.47.129:43549/ using document root <none> and password file <none>
I20260925 12:24:10.319015 15037 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:10.319056 15037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:10.319139 15037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:10.320233 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "3cd12663f94a48488454d2f2b31d0280"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "70de5d4cf23c285c3ad32c2ff4d5959a"
server_key_iv: "2f3c7e92cfa8ad7c79a0551294f4f76d"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.320542 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "3cd12663f94a48488454d2f2b31d0280"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "70de5d4cf23c285c3ad32c2ff4d5959a"
server_key_iv: "2f3c7e92cfa8ad7c79a0551294f4f76d"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.321810 15037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:24:10.322582 15052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.322772 15037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:24:10.322850 15037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "3cd12663f94a48488454d2f2b31d0280"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "70de5d4cf23c285c3ad32c2ff4d5959a"
server_key_iv: "2f3c7e92cfa8ad7c79a0551294f4f76d"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.322906 15037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:10.330338 15037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:10.332834 15037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:10.333019 15037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:10.333251 15037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:10.333598 15037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:10.333659 15037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.333704 15037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:10.333735 15037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.340000 15037 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38543
I20260925 12:24:10.340034 15165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38543 every 8 connection(s)
I20260925 12:24:10.340399 15037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:10.345533 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15037
I20260925 12:24:10.345652 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:24:10.345850 1214 external_mini_cluster.cc:1428] Setting key 5af47766d816027610f90605deffbfb0
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:10.350157 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.341154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:34019 (local address 127.1.47.190:37349)
0925 12:24:10.341239 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.341241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.341433 (+ 192us) server_negotiation.cc:407] Connection header received
0925 12:24:10.342003 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.342005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.342016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.342075 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:10.343077 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.343628 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.344625 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.344829 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.344892 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.347130 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:10.347132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:10.347132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:10.347140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:10.348611 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.349100 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.349101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.349102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.349161 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.349469 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.349470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.349470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.349620 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:10.349690 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.349953 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.350043 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 12:24:10.350385 15168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.341108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:34019)
0925 12:24:10.341286 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.341321 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.341900 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.342151 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.342154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.342730 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:10.342960 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.342967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.343725 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.343726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.344549 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.344551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.344913 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.344914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.345020 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.345553 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:10.345561 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:10.347042 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:10.348693 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.348700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.348702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.349023 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.349252 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.349253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.349253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.349391 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.349688 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:10.349692 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:10.349809 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.349896 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.350017 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260925 12:24:10.350884 15166 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:10.350976 15166 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:10.351176 15166 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:10.351703 14975 ts_manager.cc:194] Registered new tserver with Master: 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543)
I20260925 12:24:10.352522 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:34019
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:24:10.371368 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:10.456598 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:10.456841 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:10.456884 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:10.458102 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:10.458171 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:10.458217 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:10.459729 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:10.460024 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:10.460269 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:10.465531 15179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.465531 15176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.465566 15177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.465716 15171 server_base.cc:1034] running on GCE node
I20260925 12:24:10.466008 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:10.466290 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:10.467450 15171 hybrid_clock.cc:648] HybridClock initialized: now 1790339050467435 us; error 27 us; skew 500 ppm
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:10.470034 15171 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:10.470530 15171 webserver.cc:460] Webserver started at http://127.1.47.130:45835/ using document root <none> and password file <none>
I20260925 12:24:10.470768 15171 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:10.470835 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:10.470995 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:10.472144 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "91544717afc643afb794e15d1fe90bb4"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "3adbd47196bf3fa0727b743f7f27b681"
server_key_iv: "0ee1b293ba83328754664b6022bd883e"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.472517 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "91544717afc643afb794e15d1fe90bb4"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "3adbd47196bf3fa0727b743f7f27b681"
server_key_iv: "0ee1b293ba83328754664b6022bd883e"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.473843 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:10.474611 15186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.474797 15171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:24:10.474900 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "91544717afc643afb794e15d1fe90bb4"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "3adbd47196bf3fa0727b743f7f27b681"
server_key_iv: "0ee1b293ba83328754664b6022bd883e"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.474984 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:10.507296 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:10.510260 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:10.510443 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:10.510699 15171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:10.511032 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:10.511090 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.511133 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:10.511160 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.516826 15171 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:41503
I20260925 12:24:10.516865 15299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:41503 every 8 connection(s)
I20260925 12:24:10.517208 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:10.526345 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.517975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:55091 (local address 127.1.47.190:37349)
0925 12:24:10.518041 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.518043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.518257 (+ 214us) server_negotiation.cc:407] Connection header received
0925 12:24:10.518736 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.518737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.518747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.518808 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:10.519755 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.520323 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.521347 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.521559 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.521632 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.523745 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:10.523747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:10.523747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:10.523754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:10.525049 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.525521 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.525522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.525523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.525581 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.525813 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.525813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.525814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.525929 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:10.525982 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.526218 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.526280 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:24:10.526578 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.517948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:55091)
0925 12:24:10.518127 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.518169 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.518647 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.518861 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.518864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.519456 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:10.519650 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.519655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.520404 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.520404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.521218 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.521220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.521645 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.521646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.521750 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.522274 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:10.522283 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:10.523658 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:10.525119 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.525125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.525127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.525411 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.525640 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.525640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.525641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.525742 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.525987 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:10.525991 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:10.526108 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.526189 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.526306 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260925 12:24:10.526695 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15171
I20260925 12:24:10.526813 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:24:10.527032 1214 external_mini_cluster.cc:1428] Setting key 10f1fe5bbc95158a58515e15550d9cab
I20260925 12:24:10.527331 15300 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:10.527456 15300 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:10.527657 15300 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:10.528151 14975 ts_manager.cc:194] Registered new tserver with Master: 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503)
I20260925 12:24:10.528537 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:55091
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:24:10.551607 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:10.633056 15305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:10.633291 15305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:10.633329 15305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:10.634464 15305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:10.634526 15305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:10.634572 15305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:10.636126 15305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:10.636382 15305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:10.636607 15305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:10.641367 15310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.641423 15313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.641575 15305 server_base.cc:1034] running on GCE node
W20260925 12:24:10.641587 15311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.641849 15305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:10.642117 15305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:10.643277 15305 hybrid_clock.cc:648] HybridClock initialized: now 1790339050643268 us; error 31 us; skew 500 ppm
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:10.645834 15305 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:10.646320 15305 webserver.cc:460] Webserver started at http://127.1.47.131:37757/ using document root <none> and password file <none>
I20260925 12:24:10.646557 15305 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:10.646625 15305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:10.646775 15305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:10.647804 15305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "046a89c75cd5496986042b29ae07bba7"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "1ed87b61f6f7bc4a1571912ce369fe75"
server_key_iv: "ba303bbee1f23c93709d4d14f46c7101"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.648226 15305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "046a89c75cd5496986042b29ae07bba7"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "1ed87b61f6f7bc4a1571912ce369fe75"
server_key_iv: "ba303bbee1f23c93709d4d14f46c7101"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.649530 15305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:10.650308 15320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.650496 15305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:10.650593 15305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "046a89c75cd5496986042b29ae07bba7"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "1ed87b61f6f7bc4a1571912ce369fe75"
server_key_iv: "ba303bbee1f23c93709d4d14f46c7101"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.650681 15305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:10.662741 15305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:10.665222 15305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:10.665393 15305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:10.665643 15305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:10.665966 15305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:10.666019 15305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.666064 15305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:10.666091 15305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.672216 15305 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:46845
I20260925 12:24:10.672255 15433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:46845 every 8 connection(s)
I20260925 12:24:10.672610 15305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:24:10.676581 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15305
I20260925 12:24:10.676678 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:24:10.676868 1214 external_mini_cluster.cc:1428] Setting key 34f2514bdcdd96603f5bbb06c943d45f
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:10.682829 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.673366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:42807 (local address 127.1.47.190:37349)
0925 12:24:10.673444 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.673447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.673793 (+ 346us) server_negotiation.cc:407] Connection header received
0925 12:24:10.674289 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.674290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.674301 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.674361 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:10.675420 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.675998 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.677565 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.677870 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.677944 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.680349 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:10.680350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:10.680351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:10.680357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:10.681647 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.682028 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.682028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.682029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.682081 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.682296 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.682297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.682297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.682423 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:10.682477 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.682709 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.682767 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:24:10.683008 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.673365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:42807)
0925 12:24:10.673624 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.673671 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.674203 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.674441 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.674445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.675100 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:10.675313 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.675319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.676081 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.676082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.677455 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.677458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.677960 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.677961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.678108 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.678720 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:10.678728 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:10.680261 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:10.681730 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.681736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.681738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.681958 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.682138 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:10.682139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:10.682139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:10.682236 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:10.682482 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:10.682485 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:10.682598 (+ 113us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.682681 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.682800 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":112,"threads_started":1}
I20260925 12:24:10.683492 15434 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:10.683604 15434 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:10.683784 15434 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:10.684194 14975 ts_manager.cc:194] Registered new tserver with Master: 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
I20260925 12:24:10.684566 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:42807
I20260925 12:24:10.688616 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:10.695654 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.689865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.1.47.190:37349)
0925 12:24:10.689931 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.689933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.690127 (+ 194us) server_negotiation.cc:407] Connection header received
0925 12:24:10.690156 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.690157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.690167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.690211 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:10.690741 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.691198 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.691912 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.692131 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.692185 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.693334 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:10.693335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:10.693336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:10.693342 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:10.694748 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.695033 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.695033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.695034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.695090 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:10.695230 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:10.695230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:10.695230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:10.695358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:10.695442 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.695512 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.695566 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 12:24:10.697186 14975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:24:10.698782 14975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:24:10.705580 15449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.1.47.131:46845)
0925 12:24:10.702010 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.702014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.702022 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:10.702060 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.702062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.702076 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.702149 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:10.702316 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703348 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.704661 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705156 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705258 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705322 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.705447 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.705485 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":91,"threads_started":1}
I20260925 12:24:10.705621 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38543 (local address 127.0.0.1:60838)
0925 12:24:10.701435 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.701445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.701498 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.701738 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.701741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.701743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:10.701814 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.701816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703169 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.703170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.704324 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.704326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705371 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.705436 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705441 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.705492 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.705538 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":35,"thread_start_us":117,"threads_started":1}
I20260925 12:24:10.705708 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.1.47.130:41503)
0925 12:24:10.701749 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.701753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.701877 (+ 124us) server_negotiation.cc:407] Connection header received
0925 12:24:10.701961 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.701963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.701980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.702059 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:10.702266 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703036 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.704550 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705252 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705311 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705358 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.705583 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.705621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260925 12:24:10.705874 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46845 (local address 127.0.0.1:33838)
0925 12:24:10.701318 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.701363 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.701464 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.702169 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.702170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.702173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:10.702230 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.702232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703456 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.703457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.704580 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.704581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705257 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.705339 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705353 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.705426 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.705488 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260925 12:24:10.705936 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41503 (local address 127.0.0.1:40066)
0925 12:24:10.701824 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.701834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.701884 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.702141 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.702142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.702144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:10.702206 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.702208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703136 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.703137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.704459 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.704462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705386 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.705446 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705450 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.705495 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.705539 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":44,"thread_start_us":66,"threads_started":1}
I20260925 12:24:10.705945 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.701261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.1.47.129:38543)
0925 12:24:10.701561 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.701567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.701579 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:24:10.701619 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.701621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.701632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.701721 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:10.701902 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.703053 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.704422 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.705195 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.705271 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.705334 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.705800 (+ 466us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.705855 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":85,"threads_started":1}
I20260925 12:24:10.706655 15100 tablet_service.cc:1467] Processing CreateTablet for tablet a76c9c6759344ae48e5fe406a4230b12 (DEFAULT_TABLE table=test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:10.706722 15368 tablet_service.cc:1467] Processing CreateTablet for tablet a76c9c6759344ae48e5fe406a4230b12 (DEFAULT_TABLE table=test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:10.707008 15100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76c9c6759344ae48e5fe406a4230b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:10.707033 15368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76c9c6759344ae48e5fe406a4230b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:10.708333 15234 tablet_service.cc:1467] Processing CreateTablet for tablet a76c9c6759344ae48e5fe406a4230b12 (DEFAULT_TABLE table=test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:10.708670 15234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76c9c6759344ae48e5fe406a4230b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:10.710028 15451 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap starting.
I20260925 12:24:10.710378 15452 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Bootstrap starting.
I20260925 12:24:10.710842 15451 tablet_bootstrap.cc:654] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:10.711190 15452 tablet_bootstrap.cc:654] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:10.711417 15451 log.cc:824] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:10.711787 15452 log.cc:824] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:10.712368 15454 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Bootstrap starting.
I20260925 12:24:10.712599 15452 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: No bootstrap required, opened a new log
I20260925 12:24:10.712668 15452 ts_tablet_manager.cc:1397] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:10.712809 15451 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: No bootstrap required, opened a new log
I20260925 12:24:10.712883 15451 ts_tablet_manager.cc:1397] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:24:10.713011 15454 tablet_bootstrap.cc:654] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:10.713572 15454 log.cc:824] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:10.714356 15454 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: No bootstrap required, opened a new log
I20260925 12:24:10.714427 15454 ts_tablet_manager.cc:1397] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:10.714455 15452 raft_consensus.cc:348] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.714648 15452 raft_consensus.cc:374] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:10.714690 15452 raft_consensus.cc:729] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046a89c75cd5496986042b29ae07bba7, State: Initialized, Role: FOLLOWER
I20260925 12:24:10.714805 15452 consensus_queue.cc:260] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.715101 15452 ts_tablet_manager.cc:1428] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:24:10.715008 15451 raft_consensus.cc:348] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.715180 15451 raft_consensus.cc:374] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:10.715224 15451 raft_consensus.cc:729] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd12663f94a48488454d2f2b31d0280, State: Initialized, Role: FOLLOWER
I20260925 12:24:10.715233 15434 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
I20260925 12:24:10.715355 15451 consensus_queue.cc:260] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.715639 15451 ts_tablet_manager.cc:1428] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:10.715755 15166 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
I20260925 12:24:10.716919 15454 raft_consensus.cc:348] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.717047 15454 raft_consensus.cc:374] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:10.717068 15454 raft_consensus.cc:729] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91544717afc643afb794e15d1fe90bb4, State: Initialized, Role: FOLLOWER
I20260925 12:24:10.717140 15454 consensus_queue.cc:260] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.717350 15454 ts_tablet_manager.cc:1428] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:10.717386 15300 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
W20260925 12:24:10.767799 15301 tablet.cc:2369] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:24:10.781926 15458 raft_consensus.cc:482] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:10.782111 15458 raft_consensus.cc:504] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.782510 15458 leader_election.cc:288] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845), 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503)
I20260925 12:24:10.785476 15449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.782895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:56101 (local address 127.1.47.131:46845)
0925 12:24:10.783003 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.783005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.783013 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:10.783044 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.783045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.783060 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.783132 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:10.783291 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.783766 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.784734 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.785118 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.785208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.785217 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.785374 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.785412 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 12:24:10.785482 15168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.782756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46845 (local address 127.1.47.129:56101)
0925 12:24:10.782816 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.782829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.782877 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.783140 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.783141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.783143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:10.783218 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.783220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.783883 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.783884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.784640 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.784641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.785204 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.785205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.785292 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.785295 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.785356 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.785402 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 12:24:10.785770 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.782970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:58219 (local address 127.1.47.130:41503)
0925 12:24:10.783034 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.783037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.783164 (+ 127us) server_negotiation.cc:407] Connection header received
0925 12:24:10.783248 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.783248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.783260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.783331 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:10.783524 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.784071 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.784988 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.785369 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.785460 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.785467 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.785659 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.785708 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:24:10.785877 15388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76c9c6759344ae48e5fe406a4230b12" candidate_uuid: "3cd12663f94a48488454d2f2b31d0280" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046a89c75cd5496986042b29ae07bba7" is_pre_election: true
I20260925 12:24:10.786036 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.782884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41503 (local address 127.1.47.129:58219)
0925 12:24:10.783082 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.783091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:10.783179 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:10.783335 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:10.783336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:10.783338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:10.783441 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.783443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.784152 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.784153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.784890 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:10.784892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.785459 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.785460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:10.785559 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.785564 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:24:10.785886 (+ 322us) client_negotiation.cc:240] Negotiation successful
0925 12:24:10.785934 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260925 12:24:10.786047 15388 raft_consensus.cc:2454] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd12663f94a48488454d2f2b31d0280 in term 0.
I20260925 12:24:10.786319 15055 leader_election.cc:302] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 046a89c75cd5496986042b29ae07bba7, 3cd12663f94a48488454d2f2b31d0280; no voters:
I20260925 12:24:10.786350 15254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76c9c6759344ae48e5fe406a4230b12" candidate_uuid: "3cd12663f94a48488454d2f2b31d0280" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91544717afc643afb794e15d1fe90bb4" is_pre_election: true
I20260925 12:24:10.786466 15458 raft_consensus.cc:2781] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:10.786517 15458 raft_consensus.cc:482] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:10.786512 15254 raft_consensus.cc:2454] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd12663f94a48488454d2f2b31d0280 in term 0.
I20260925 12:24:10.786553 15458 raft_consensus.cc:3037] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:10.787283 15458 raft_consensus.cc:504] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.787429 15458 leader_election.cc:288] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 1 election: Requested vote from peers 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845), 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503)
I20260925 12:24:10.787607 15388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76c9c6759344ae48e5fe406a4230b12" candidate_uuid: "3cd12663f94a48488454d2f2b31d0280" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046a89c75cd5496986042b29ae07bba7"
I20260925 12:24:10.787621 15254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76c9c6759344ae48e5fe406a4230b12" candidate_uuid: "3cd12663f94a48488454d2f2b31d0280" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91544717afc643afb794e15d1fe90bb4"
I20260925 12:24:10.787690 15388 raft_consensus.cc:3037] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:10.787690 15254 raft_consensus.cc:3037] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:10.788614 15388 raft_consensus.cc:2454] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd12663f94a48488454d2f2b31d0280 in term 1.
I20260925 12:24:10.788638 15254 raft_consensus.cc:2454] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd12663f94a48488454d2f2b31d0280 in term 1.
I20260925 12:24:10.788792 15055 leader_election.cc:302] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 046a89c75cd5496986042b29ae07bba7, 3cd12663f94a48488454d2f2b31d0280; no voters:
I20260925 12:24:10.788897 15458 raft_consensus.cc:2781] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:10.789050 15458 raft_consensus.cc:686] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 LEADER]: Becoming Leader. State: Replica: 3cd12663f94a48488454d2f2b31d0280, State: Running, Role: LEADER
I20260925 12:24:10.789146 15458 consensus_queue.cc:237] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:10.789978 14974 catalog_manager.cc:5495] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "3cd12663f94a48488454d2f2b31d0280" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } health_report { overall_health: UNKNOWN } } }
I20260925 12:24:10.796950 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.794661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.1.47.129:38543)
0925 12:24:10.794701 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:10.794703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:10.794711 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:10.794748 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:10.794749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:10.794765 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:10.794821 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:10.794926 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.795409 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.796203 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:10.796392 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:10.796477 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:10.796613 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:10.796713 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:10.796776 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:10.796857 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 12:24:10.796874 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:24:10.798730 15254 raft_consensus.cc:1261] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Refusing update from remote peer 3cd12663f94a48488454d2f2b31d0280: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:10.798835 15388 raft_consensus.cc:1261] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Refusing update from remote peer 3cd12663f94a48488454d2f2b31d0280: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:10.799115 15461 consensus_queue.cc:1035] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:10.799273 15458 consensus_queue.cc:1035] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [LEADER]: Connected to new peer: Peer: permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:10.803095 15466 mvcc.cc:205] Tried to move back new op lower bound from 7333228752068898816 to 7333228752032903168. Current Snapshot: MvccSnapshot[applied={T|T < 7333228752068898816}]
I20260925 12:24:10.804662 14975 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:10.804749 14975 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:10.805486 14975 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Sending DeleteTablet for 3 replicas of tablet a76c9c6759344ae48e5fe406a4230b12
I20260925 12:24:10.805850 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15037
I20260925 12:24:10.805836 15368 tablet_service.cc:1514] Processing DeleteTablet for tablet a76c9c6759344ae48e5fe406a4230b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:10 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:33838
I20260925 12:24:10.806442 15234 tablet_service.cc:1514] Processing DeleteTablet for tablet a76c9c6759344ae48e5fe406a4230b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:10 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40066
I20260925 12:24:10.806490 15474 tablet_replica.cc:330] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: stopping tablet replica
I20260925 12:24:10.806602 15474 raft_consensus.cc:2229] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:10.806692 15475 tablet_replica.cc:330] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: stopping tablet replica
I20260925 12:24:10.806770 15474 raft_consensus.cc:2258] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:10.806809 15475 raft_consensus.cc:2229] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:10.806937 15475 raft_consensus.cc:2258] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:10.807427 15475 ts_tablet_manager.cc:1905] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:10.808804 15474 ts_tablet_manager.cc:1905] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:10.809459 15475 ts_tablet_manager.cc:1918] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:10.809567 15475 log.cc:1196] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/a76c9c6759344ae48e5fe406a4230b12
I20260925 12:24:10.809952 15475 ts_tablet_manager.cc:1939] T a76c9c6759344ae48e5fe406a4230b12 P 91544717afc643afb794e15d1fe90bb4: Deleting consensus metadata
I20260925 12:24:10.810416 14961 catalog_manager.cc:4841] TS 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): tablet a76c9c6759344ae48e5fe406a4230b12 (table test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]) successfully deleted
I20260925 12:24:10.810576 15474 ts_tablet_manager.cc:1918] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:10.810686 15474 log.cc:1196] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/a76c9c6759344ae48e5fe406a4230b12
I20260925 12:24:10.810964 15474 ts_tablet_manager.cc:1939] T a76c9c6759344ae48e5fe406a4230b12 P 046a89c75cd5496986042b29ae07bba7: Deleting consensus metadata
I20260925 12:24:10.811409 14961 catalog_manager.cc:4841] TS 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): tablet a76c9c6759344ae48e5fe406a4230b12 (table test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]) successfully deleted
W20260925 12:24:10.812419 14959 connection.cc:535] client connection to 127.1.47.129:38543 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 12:24:10.812551 14959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 12:24:10.812654 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15171
I20260925 12:24:10.813974 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:10.812960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38543 (local address 127.0.0.1:60854)
0925 12:24:10.813900 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:10.813944 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":918}
W20260925 12:24:10.814141 14959 catalog_manager.cc:4584] TS 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a76c9c6759344ae48e5fe406a4230b12: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38543: connect: Connection refused (error 111)
I20260925 12:24:10.819029 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15305
I20260925 12:24:10.823606 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 14943
I20260925 12:24:10.829532 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--webserver_interface=127.1.47.190
--webserver_port=41521
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:10.911715 15477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:10.912062 15477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:10.912106 15477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:10.913306 15477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:10.913385 15477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:10.913409 15477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:10.913430 15477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:10.913460 15477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:10.915021 15477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=41521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:10.915350 15477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:10.915592 15477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:10.924383 15483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.924429 15486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:10.924393 15484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:10.924649 15477 server_base.cc:1034] running on GCE node
I20260925 12:24:10.924929 15477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:10.925254 15477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:10.926395 15477 hybrid_clock.cc:648] HybridClock initialized: now 1790339050926377 us; error 42 us; skew 500 ppm
Sep 25 12:24:10 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:10.928881 15477 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:10.929327 15477 webserver.cc:460] Webserver started at http://127.1.47.190:41521/ using document root <none> and password file <none>
I20260925 12:24:10.929590 15477 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:10.929648 15477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:10.931098 15477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:24:10.931802 15493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:10.932013 15477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:10.932103 15477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "0abca66feadb45bea1ba04e67154adb0"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "65ba65e4beb4a0a89c1ad3fe6e3519ae"
server_key_iv: "641306946145e32435ec6b8aa772abd9"
server_key_version: "encryptionkey@0"
I20260925 12:24:10.932502 15477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:10.957829 15477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:10.960465 15477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:10.960598 15477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:10.964856 15477 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37349
I20260925 12:24:10.964934 15545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37349 every 8 connection(s)
I20260925 12:24:10.965358 15477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:10.967036 15546 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:24:10.967700 15546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap starting.
I20260925 12:24:10.974865 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15477
I20260925 12:24:10.975203 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:38543
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=43549
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:10.976080 15546 log.cc:824] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:10.978056 15546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:10.978386 15546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap complete.
I20260925 12:24:10.980784 15546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.981048 15546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Initialized, Role: FOLLOWER
I20260925 12:24:10.981258 15546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.981355 15546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:10.981415 15546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:10.981467 15546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:24:10.982494 15546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.982616 15546 leader_election.cc:302] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0abca66feadb45bea1ba04e67154adb0; no voters:
I20260925 12:24:10.982833 15546 leader_election.cc:288] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 12:24:10.982868 15550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 12:24:10.982982 15550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 LEADER]: Becoming Leader. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Running, Role: LEADER
I20260925 12:24:10.983075 15550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:10.983119 15546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:10.983357 15550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0abca66feadb45bea1ba04e67154adb0. Latest consensus state: current_term: 2 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:10.983433 15550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:10.983364 15552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:10.983500 15552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:10.984117 15554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:10.985100 15554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:10.985289 15554 catalog_manager.cc:1259] Loaded cluster ID: 5cfcd29934de46fea103f9bfbf349785
I20260925 12:24:10.985503 15554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:10.985790 15554 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 12:24:10.990547 15554 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 12:24:10.993018 15554 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:10.993206 15554 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Loaded TSK: 0
I20260925 12:24:10.993427 15554 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 12:24:11.062721 15548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.062949 15548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.062978 15548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.064466 15548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.064520 15548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.064554 15548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:11.066215 15548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:38543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=43549
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.066470 15548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.066723 15548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.071815 15574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.071852 15573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.072077 15576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.072074 15548 server_base.cc:1034] running on GCE node
I20260925 12:24:11.072405 15548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.072692 15548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.073871 15548 hybrid_clock.cc:648] HybridClock initialized: now 1790339051073826 us; error 59 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.076516 15548 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:11.076974 15548 webserver.cc:460] Webserver started at http://127.1.47.129:43549/ using document root <none> and password file <none>
I20260925 12:24:11.077203 15548 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.077258 15548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.078715 15548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.079485 15583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.079689 15548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.079800 15548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "3cd12663f94a48488454d2f2b31d0280"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "70de5d4cf23c285c3ad32c2ff4d5959a"
server_key_iv: "2f3c7e92cfa8ad7c79a0551294f4f76d"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.080227 15548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.099525 15548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.101824 15548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.102010 15548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.102283 15548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:11.102797 15590 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 12:24:11.104364 15548 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 12:24:11.104422 15548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 12:24:11.104455 15548 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 12:24:11.105099 15548 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 12:24:11.105144 15548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 12:24:11.105186 15590 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap starting.
I20260925 12:24:11.112849 15548 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38543
I20260925 12:24:11.113026 15697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38543 every 8 connection(s)
I20260925 12:24:11.113415 15548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:11.116695 15590 log.cc:824] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:11.119226 15590 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:11.119550 15590 tablet_bootstrap.cc:492] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap complete.
I20260925 12:24:11.120641 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15548
I20260925 12:24:11.120769 15590 ts_tablet_manager.cc:1397] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260925 12:24:11.120937 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:41503
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=45835
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:11.122198 15590 raft_consensus.cc:348] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:11.122503 15590 raft_consensus.cc:729] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd12663f94a48488454d2f2b31d0280, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.122650 15590 consensus_queue.cc:260] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } } peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } }
I20260925 12:24:11.122972 15590 ts_tablet_manager.cc:1428] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 12:24:11.126308 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.114301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:35231 (local address 127.1.47.190:37349)
0925 12:24:11.114440 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.114450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.114487 (+ 37us) server_negotiation.cc:407] Connection header received
0925 12:24:11.115448 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.115455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.115502 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.115611 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:11.116631 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.117571 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.119006 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.119472 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.119558 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.122252 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:11.122256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:11.122257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:11.122270 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:11.124339 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.124870 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.124873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.124873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.124965 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.125309 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.125310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.125310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.125583 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:11.125666 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.126035 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.126157 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260925 12:24:11.126442 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.114132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:35231)
0925 12:24:11.114343 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.114405 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.115232 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.115709 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.115715 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.116196 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:11.116488 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.116497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.117683 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.117684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.118916 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.118919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.119575 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.119577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.119724 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.120284 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:11.120294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:11.122129 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:11.124432 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.124439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.124441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.124782 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.125040 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.125042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.125042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.125195 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.125667 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:11.125674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:11.125852 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.125983 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.126181 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":110,"threads_started":1}
I20260925 12:24:11.127369 15698 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:11.127486 15698 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:11.127687 15698 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:11.128750 15510 ts_manager.cc:194] Registered new tserver with Master: 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543)
I20260925 12:24:11.130146 15510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:35231
I20260925 12:24:11.131307 15698 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339050, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.129@KRBTEST.COM
I20260925 12:24:11.138753 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.129586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38543 (local address 127.0.0.1:60862)
0925 12:24:11.129856 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.129891 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.129988 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.130201 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.130206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.130642 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:11.130712 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.130714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.131670 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.131671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.132351 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.132354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.132945 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.132946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.133039 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.133644 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:11.133650 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:11.135298 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:11.137349 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.137356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.137358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.137586 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.137879 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.137880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.137880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.137963 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.138316 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:11.138323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:11.138382 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.138496 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.138646 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260925 12:24:11.138751 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.129635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.1.47.129:38543)
0925 12:24:11.129787 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.129791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.129966 (+ 175us) server_negotiation.cc:407] Connection header received
0925 12:24:11.130090 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.130093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.130109 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.130199 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:11.130801 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.131586 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.132455 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.132815 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.132880 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.135386 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:11.135395 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:11.135395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:11.135403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:11.137234 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.137727 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.137730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.137730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.137791 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.138051 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.138052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.138052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.138227 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:11.138329 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.138535 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.138638 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260925 12:24:11.139271 15632 tablet_service.cc:1514] Processing DeleteTablet for tablet a76c9c6759344ae48e5fe406a4230b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:10 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:60862
I20260925 12:24:11.139919 15707 tablet_replica.cc:330] stopping tablet replica
I20260925 12:24:11.140035 15707 raft_consensus.cc:2229] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:11.140102 15707 raft_consensus.cc:2258] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:11.140479 15707 ts_tablet_manager.cc:1905] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:11.142863 15707 ts_tablet_manager.cc:1918] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:11.142948 15707 log.cc:1196] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/a76c9c6759344ae48e5fe406a4230b12
I20260925 12:24:11.143173 15707 ts_tablet_manager.cc:1939] T a76c9c6759344ae48e5fe406a4230b12 P 3cd12663f94a48488454d2f2b31d0280: Deleting consensus metadata
I20260925 12:24:11.143766 15494 catalog_manager.cc:4841] TS 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543): tablet a76c9c6759344ae48e5fe406a4230b12 (table test-table [id=c81c899d19e245bd8bae6cdd5f852a5e]) successfully deleted
W20260925 12:24:11.207803 15703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.208041 15703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.208066 15703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.209285 15703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.209328 15703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.209355 15703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:11.210800 15703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:41503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=45835
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.211068 15703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.211308 15703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.216282 15715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.216286 15713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.216415 15712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.216575 15703 server_base.cc:1034] running on GCE node
I20260925 12:24:11.216832 15703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.217108 15703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.218300 15703 hybrid_clock.cc:648] HybridClock initialized: now 1790339051218297 us; error 46 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.221197 15703 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:11.221722 15703 webserver.cc:460] Webserver started at http://127.1.47.130:45835/ using document root <none> and password file <none>
I20260925 12:24:11.221982 15703 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.222083 15703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.223629 15703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:24:11.224349 15722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.224531 15703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.224644 15703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "91544717afc643afb794e15d1fe90bb4"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "3adbd47196bf3fa0727b743f7f27b681"
server_key_iv: "0ee1b293ba83328754664b6022bd883e"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.225045 15703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.245364 15703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.247509 15703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.247699 15703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.247992 15703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:11.248329 15703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:11.248391 15703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.248437 15703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:11.248484 15703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.254760 15703 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:41503
I20260925 12:24:11.254833 15835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:41503 every 8 connection(s)
I20260925 12:24:11.255244 15703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:11.256597 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15703
I20260925 12:24:11.256930 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:46845
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=37757
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:11.265823 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.256017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:53247)
0925 12:24:11.256217 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.256272 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.256826 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.257124 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.257128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.257740 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:11.257942 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.257947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.258667 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.258668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.259489 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.259492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.260040 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.260041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.260161 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.260790 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:11.260801 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:11.262274 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:11.264327 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.264334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.264336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.264682 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.264894 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.264895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.264895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.265003 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.265310 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:11.265314 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:11.265433 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.265533 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.265656 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260925 12:24:11.265825 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.256025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:53247 (local address 127.1.47.190:37349)
0925 12:24:11.256112 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.256114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.256378 (+ 264us) server_negotiation.cc:407] Connection header received
0925 12:24:11.256943 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.256945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.256960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.257037 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:11.258054 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.258568 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.259586 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.259942 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.260016 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.262372 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:11.262373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:11.262374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:11.262384 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:11.264243 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.264746 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.264747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.264748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.264818 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.265073 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.265074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.265074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.265238 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:11.265317 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.265584 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.265687 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 12:24:11.266530 15836 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:11.266623 15836 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:11.266819 15836 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:11.267242 15510 ts_manager.cc:194] Registered new tserver with Master: 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503)
I20260925 12:24:11.267629 15510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:53247
W20260925 12:24:11.341107 15839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.341343 15839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.341380 15839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.342532 15839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.342599 15839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.342645 15839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:11.344199 15839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:46845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=37757
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.344478 15839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.344719 15839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.349686 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.349761 15839 server_base.cc:1034] running on GCE node
W20260925 12:24:11.349701 15844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.349701 15845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.350135 15839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.350332 15839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.351482 15839 hybrid_clock.cc:648] HybridClock initialized: now 1790339051351470 us; error 27 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.353873 15839 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:11.354269 15839 webserver.cc:460] Webserver started at http://127.1.47.131:37757/ using document root <none> and password file <none>
I20260925 12:24:11.354528 15839 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.354583 15839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.355940 15839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.356482 15854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.356650 15839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.356753 15839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "046a89c75cd5496986042b29ae07bba7"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "1ed87b61f6f7bc4a1571912ce369fe75"
server_key_iv: "ba303bbee1f23c93709d4d14f46c7101"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.357101 15839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.365401 15839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.367723 15839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.367918 15839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.368180 15839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:11.368511 15839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:11.368546 15839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.368593 15839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:11.368624 15839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.374788 15839 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:46845
I20260925 12:24:11.374840 15967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:46845 every 8 connection(s)
I20260925 12:24:11.375221 15839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:11.382819 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 15839
I20260925 12:24:11.385164 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.375995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:48565 (local address 127.1.47.190:37349)
0925 12:24:11.376068 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.376070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.376278 (+ 208us) server_negotiation.cc:407] Connection header received
0925 12:24:11.376775 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.376776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.376787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.376838 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:11.377847 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.378326 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.379418 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.379613 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.379668 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.381854 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:11.381856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:11.381856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:11.381863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:11.383562 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.384139 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.384140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.384141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.384205 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.384472 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.384473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.384473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.384581 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:11.384667 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.384982 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.385068 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 12:24:11.385371 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.375985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:48565)
0925 12:24:11.376153 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.376191 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.376689 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.376897 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.376901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.377555 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:11.377767 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.377772 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.378418 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.378418 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.379299 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.379301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.379690 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.379690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.379794 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.380370 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:11.380379 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:11.381760 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:11.383671 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.383680 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.383682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.384028 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.384277 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:11.384278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:11.384279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:11.384402 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:11.384658 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:11.384665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:11.384822 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.384944 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.385118 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260925 12:24:11.385934 15968 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37349
I20260925 12:24:11.386052 15968 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:11.386246 15968 heartbeater.cc:511] Master 127.1.47.190:37349 requested a full tablet report, sending...
I20260925 12:24:11.386713 15510 ts_manager.cc:194] Registered new tserver with Master: 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
I20260925 12:24:11.387235 15510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:48565
I20260925 12:24:11.390300 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.383261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.1.47.190:37349)
0925 12:24:11.383466 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.383468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.383476 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:11.383514 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.383514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.383526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.383582 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:11.384471 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.384943 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.386030 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.386283 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.386364 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.387773 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:11.387774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:11.387774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:11.387782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:11.389149 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.389520 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.389521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.389522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.389581 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:11.389792 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:11.389793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:11.389793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:11.389906 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:11.390086 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.390122 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.390197 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260925 12:24:11.391041 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:11.393572 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.391357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.1.47.190:37349)
0925 12:24:11.391415 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.391417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.391523 (+ 106us) server_negotiation.cc:407] Connection header received
0925 12:24:11.391567 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.391568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.391593 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.391649 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:11.391737 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.392249 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.392967 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.393147 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.393195 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.393318 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:11.393408 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:11.393468 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.393509 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.393527 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:24:11.393815 15510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:24:11.395391 15510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:24:11.397687 15632 tablet_service.cc:1467] Processing CreateTablet for tablet dc51c7dd2a8a495aab6473635a119480 (DEFAULT_TABLE table=test-table [id=239d3600155b41a981b64999828d7504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:11.397889 15632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51c7dd2a8a495aab6473635a119480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:11.399416 15590 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap starting.
I20260925 12:24:11.399768 15590 tablet_bootstrap.cc:654] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:11.400394 15590 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: No bootstrap required, opened a new log
I20260925 12:24:11.400441 15590 ts_tablet_manager.cc:1397] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.400767 15590 raft_consensus.cc:348] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.400852 15590 raft_consensus.cc:374] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:11.400895 15590 raft_consensus.cc:729] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd12663f94a48488454d2f2b31d0280, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.400977 15590 consensus_queue.cc:260] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.401072 15590 ts_tablet_manager.cc:1428] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.401863 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.397461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41503 (local address 127.0.0.1:40072)
0925 12:24:11.397498 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.397521 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.397559 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.397948 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.397949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.397951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:11.398010 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.398012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.399226 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.399227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.400562 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.400564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.401579 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.401581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.401662 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.401687 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.401728 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.401780 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 12:24:11.401957 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.397560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46845 (local address 127.0.0.1:33842)
0925 12:24:11.397712 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.397722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.397767 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.398326 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.398327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.398329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:11.398367 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.398369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.399699 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.399700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.400765 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.400768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.401695 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.401696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.401746 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.401749 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.401814 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.401857 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260925 12:24:11.401969 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.397809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.1.47.131:46845)
0925 12:24:11.398168 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.398172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.398180 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:11.398210 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.398211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.398224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.398312 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:11.398486 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.399590 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.400852 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.401618 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.401680 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.401746 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.401834 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.401879 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":119,"threads_started":1}
I20260925 12:24:11.402245 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.397572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40072 (local address 127.1.47.130:41503)
0925 12:24:11.397764 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.397770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.397780 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:24:11.397820 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.397822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.397838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.397931 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:11.398076 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.399114 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.400659 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.401469 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.401982 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.402069 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.402104 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.402162 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
I20260925 12:24:11.402655 15902 tablet_service.cc:1467] Processing CreateTablet for tablet dc51c7dd2a8a495aab6473635a119480 (DEFAULT_TABLE table=test-table [id=239d3600155b41a981b64999828d7504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:11.403007 15902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51c7dd2a8a495aab6473635a119480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:11.402971 15770 tablet_service.cc:1467] Processing CreateTablet for tablet dc51c7dd2a8a495aab6473635a119480 (DEFAULT_TABLE table=test-table [id=239d3600155b41a981b64999828d7504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:11.403309 15770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51c7dd2a8a495aab6473635a119480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:11.406785 15978 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Bootstrap starting.
I20260925 12:24:11.407253 15979 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Bootstrap starting.
I20260925 12:24:11.407409 15978 tablet_bootstrap.cc:654] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:11.408039 15978 log.cc:824] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:11.408080 15979 tablet_bootstrap.cc:654] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:11.408634 15979 log.cc:824] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:11.408763 15978 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: No bootstrap required, opened a new log
I20260925 12:24:11.408833 15978 ts_tablet_manager.cc:1397] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:11.409394 15979 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: No bootstrap required, opened a new log
I20260925 12:24:11.409482 15979 ts_tablet_manager.cc:1397] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:11.410315 15978 raft_consensus.cc:348] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.410528 15978 raft_consensus.cc:374] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:11.410576 15978 raft_consensus.cc:729] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046a89c75cd5496986042b29ae07bba7, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.410744 15978 consensus_queue.cc:260] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.411038 15978 ts_tablet_manager.cc:1428] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:11.411052 15968 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
I20260925 12:24:11.411422 15979 raft_consensus.cc:348] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.411620 15979 raft_consensus.cc:374] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:11.411675 15979 raft_consensus.cc:729] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91544717afc643afb794e15d1fe90bb4, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.411811 15979 consensus_queue.cc:260] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.412156 15979 ts_tablet_manager.cc:1428] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:11.412165 15836 heartbeater.cc:503] Master 127.1.47.190:37349 was elected leader, sending a full tablet report...
I20260925 12:24:11.415091 15983 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:11.415161 15983 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.415445 15983 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845), 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543)
I20260925 12:24:11.418346 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.415580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46845 (local address 127.1.47.130:33847)
0925 12:24:11.415625 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.415638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.415684 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.415892 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.415893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.415895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:11.415955 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.415956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.416646 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.416647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.417376 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.417378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.418086 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.418087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.418174 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.418178 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.418229 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.418273 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":1}
I20260925 12:24:11.418514 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.415589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38543 (local address 127.1.47.130:35697)
0925 12:24:11.415772 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.415783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.415825 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.416010 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:11.416010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:11.416012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:11.416068 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.416069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.416697 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.416698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.417452 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:11.417454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.418286 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.418287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:11.418352 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.418355 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:11.418411 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 12:24:11.418450 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":44,"thread_start_us":91,"threads_started":1}
I20260925 12:24:11.418573 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.415683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:33847 (local address 127.1.47.131:46845)
0925 12:24:11.415721 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.415723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.415732 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:24:11.415756 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.415757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.415767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.415825 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:11.416060 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.416561 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.417470 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.417974 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.418075 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.418085 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.418455 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.418499 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:24:11.419014 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.415782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:35697 (local address 127.1.47.129:38543)
0925 12:24:11.415841 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.415843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.415851 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:11.415899 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.415899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.415930 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.415988 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:11.416132 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.416609 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.417556 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.418096 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.418183 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.418254 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.418850 (+ 596us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.418906 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:24:11.419246 15652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51c7dd2a8a495aab6473635a119480" candidate_uuid: "91544717afc643afb794e15d1fe90bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd12663f94a48488454d2f2b31d0280" is_pre_election: true
I20260925 12:24:11.419411 15652 raft_consensus.cc:2454] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91544717afc643afb794e15d1fe90bb4 in term 0.
I20260925 12:24:11.419556 15922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51c7dd2a8a495aab6473635a119480" candidate_uuid: "91544717afc643afb794e15d1fe90bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046a89c75cd5496986042b29ae07bba7" is_pre_election: true
I20260925 12:24:11.419675 15922 raft_consensus.cc:2454] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91544717afc643afb794e15d1fe90bb4 in term 0.
I20260925 12:24:11.419623 15723 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280, 91544717afc643afb794e15d1fe90bb4; no voters:
I20260925 12:24:11.419759 15983 raft_consensus.cc:2781] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:11.419812 15983 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:11.419847 15983 raft_consensus.cc:3037] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:11.420537 15983 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.420742 15983 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [CANDIDATE]: Term 1 election: Requested vote from peers 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845), 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543)
I20260925 12:24:11.420805 15652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51c7dd2a8a495aab6473635a119480" candidate_uuid: "91544717afc643afb794e15d1fe90bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd12663f94a48488454d2f2b31d0280"
I20260925 12:24:11.420907 15652 raft_consensus.cc:3037] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:11.421639 15652 raft_consensus.cc:2454] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91544717afc643afb794e15d1fe90bb4 in term 1.
I20260925 12:24:11.421772 15922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51c7dd2a8a495aab6473635a119480" candidate_uuid: "91544717afc643afb794e15d1fe90bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "046a89c75cd5496986042b29ae07bba7"
I20260925 12:24:11.421855 15922 raft_consensus.cc:3037] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:11.421864 15723 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280, 91544717afc643afb794e15d1fe90bb4; no voters:
I20260925 12:24:11.421948 15983 raft_consensus.cc:2781] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:11.422102 15983 raft_consensus.cc:686] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 91544717afc643afb794e15d1fe90bb4, State: Running, Role: LEADER
I20260925 12:24:11.422205 15983 consensus_queue.cc:237] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.422716 15922 raft_consensus.cc:2454] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91544717afc643afb794e15d1fe90bb4 in term 1.
I20260925 12:24:11.423070 15510 catalog_manager.cc:5495] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91544717afc643afb794e15d1fe90bb4 (127.1.47.130). New cstate: current_term: 1 leader_uuid: "91544717afc643afb794e15d1fe90bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } health_report { overall_health: UNKNOWN } } }
I20260925 12:24:11.428643 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.426372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.1.47.130:41503)
0925 12:24:11.426417 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:11.426420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:11.426427 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:24:11.426498 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:11.426498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:11.426512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:11.426572 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:11.426661 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.427110 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.427928 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:11.428166 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:11.428217 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:11.428336 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:11.428424 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:11.428481 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:11.428510 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 12:24:11.428526 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:24:11.430375 15922 raft_consensus.cc:1261] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Refusing update from remote peer 91544717afc643afb794e15d1fe90bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:11.430401 15652 raft_consensus.cc:1261] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Refusing update from remote peer 91544717afc643afb794e15d1fe90bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:11.430735 15983 consensus_queue.cc:1035] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:11.430894 15985 consensus_queue.cc:1035] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:11.433815 15989 mvcc.cc:205] Tried to move back new op lower bound from 7333228754656137216 to 7333228754626179072. Current Snapshot: MvccSnapshot[applied={T|T < 7333228754656137216 or (T in {7333228754656137216})}]
I20260925 12:24:11.435298 15510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:11.435388 15510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:11.436074 15510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Sending DeleteTablet for 3 replicas of tablet dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:11.436340 15770 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51c7dd2a8a495aab6473635a119480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:11 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40072
I20260925 12:24:11.436369 15632 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51c7dd2a8a495aab6473635a119480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:11 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:60862
I20260925 12:24:11.436403 15902 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51c7dd2a8a495aab6473635a119480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:11 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:33842
I20260925 12:24:11.436460 15707 tablet_replica.cc:330] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: stopping tablet replica
I20260925 12:24:11.436508 15707 raft_consensus.cc:2229] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:11.436556 15707 raft_consensus.cc:2258] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:11.436793 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15548
I20260925 12:24:11.436858 15707 ts_tablet_manager.cc:1905] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:11.439982 15999 tablet_replica.cc:330] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: stopping tablet replica
I20260925 12:24:11.440107 15999 raft_consensus.cc:2229] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:11.440225 15999 raft_consensus.cc:2258] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:11.440706 15999 ts_tablet_manager.cc:1905] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:11.442711 15999 ts_tablet_manager.cc:1918] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:11.442790 15999 log.cc:1196] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:11.443060 15999 ts_tablet_manager.cc:1939] T dc51c7dd2a8a495aab6473635a119480 P 046a89c75cd5496986042b29ae07bba7: Deleting consensus metadata
I20260925 12:24:11.443526 15496 catalog_manager.cc:4841] TS 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): tablet dc51c7dd2a8a495aab6473635a119480 (table test-table [id=239d3600155b41a981b64999828d7504]) successfully deleted
W20260925 12:24:11.443706 15494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:38543 (error 108)
I20260925 12:24:11.443928 15998 tablet_replica.cc:330] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: stopping tablet replica
I20260925 12:24:11.444084 15998 raft_consensus.cc:2229] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:24:11.444206 15998 raft_consensus.cc:2258] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 12:24:11.444468 15974 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:24:11.444186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:38543 (local address 127.0.0.1:60868)
0925 12:24:11.444241 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.444253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.444295 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.444413 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38543: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:24:11.444684 15998 ts_tablet_manager.cc:1905] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:24:11.444583 15494 catalog_manager.cc:4584] TS 3cd12663f94a48488454d2f2b31d0280 (127.1.47.129:38543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc51c7dd2a8a495aab6473635a119480: Network error: Client connection negotiation failed: client connection to 127.1.47.129:38543: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 12:24:11.445406 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15703
I20260925 12:24:11.451152 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.451032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41503 (local address 127.0.0.1:40090)
0925 12:24:11.451082 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.451123 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.130:41503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:24:11.451272 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15839
I20260925 12:24:11.455730 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 15477
I20260925 12:24:11.461114 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--webserver_interface=127.1.47.190
--webserver_port=41521
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:11.542541 16001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.542817 16001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.542857 16001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.544086 16001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:11.544167 16001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.544193 16001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.544222 16001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:11.544253 16001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:11.545763 16001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=41521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.546057 16001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.546303 16001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.551090 16009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.551131 16007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.551112 16006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.551365 16001 server_base.cc:1034] running on GCE node
I20260925 12:24:11.551616 16001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.551956 16001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.553130 16001 hybrid_clock.cc:648] HybridClock initialized: now 1790339051553110 us; error 31 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.555930 16001 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:11.556413 16001 webserver.cc:460] Webserver started at http://127.1.47.190:41521/ using document root <none> and password file <none>
I20260925 12:24:11.556655 16001 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.556725 16001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.558126 16001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.558836 16016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.559044 16001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.559144 16001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "0abca66feadb45bea1ba04e67154adb0"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "65ba65e4beb4a0a89c1ad3fe6e3519ae"
server_key_iv: "641306946145e32435ec6b8aa772abd9"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.559525 16001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.603899 16001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.606295 16001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.606474 16001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.610674 16001 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37349
I20260925 12:24:11.610719 16068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37349 every 8 connection(s)
I20260925 12:24:11.611140 16001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:11.612891 16069 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:24:11.613557 16069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap starting.
I20260925 12:24:11.616590 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16001
I20260925 12:24:11.616883 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:38543
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=43549
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:11.621868 16069 log.cc:824] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:11.624280 16069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:11.624606 16069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: Bootstrap complete.
I20260925 12:24:11.626881 16069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:11.627143 16069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.627337 16069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:11.627419 16069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:11.627497 16069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:11.627550 16069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 2 FOLLOWER]: Advancing to term 3
I20260925 12:24:11.628667 16069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:11.628789 16069 leader_election.cc:302] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0abca66feadb45bea1ba04e67154adb0; no voters:
I20260925 12:24:11.629000 16069 leader_election.cc:288] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 12:24:11.629071 16073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 12:24:11.629199 16073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [term 3 LEADER]: Becoming Leader. State: Replica: 0abca66feadb45bea1ba04e67154adb0, State: Running, Role: LEADER
I20260925 12:24:11.629285 16069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:11.629294 16073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } }
I20260925 12:24:11.629593 16073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0abca66feadb45bea1ba04e67154adb0. Latest consensus state: current_term: 3 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:11.629675 16073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:11.629597 16075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0abca66feadb45bea1ba04e67154adb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0abca66feadb45bea1ba04e67154adb0" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37349 } } }
I20260925 12:24:11.629834 16075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:11.631228 16089 catalog_manager.cc:1259] Loaded cluster ID: 5cfcd29934de46fea103f9bfbf349785
I20260925 12:24:11.631285 16089 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: loading cluster ID for follower catalog manager: success
W20260925 12:24:11.631512 16089 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0abca66feadb45bea1ba04e67154adb0: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260925 12:24:11.631557 16089 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260925 12:24:11.631596 16078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:11.632020 16078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:11.632141 16078 catalog_manager.cc:1259] Loaded cluster ID: 5cfcd29934de46fea103f9bfbf349785
I20260925 12:24:11.632177 16078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 12:24:11.632313 16078 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd0d87b89ad google::LogMessage::Fail() at ??:0
@ 0x7fd0d87bb88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd0d87b84de google::LogMessage::Flush() at ??:0
@ 0x7fd0d87b9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd0e06d90f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd0e06e28b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd0da6860d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd0da67b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd0db4df6db start_thread at ??:0
@ 0x7fd0d72b661f clone at ??:0
W20260925 12:24:11.712172 16071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.712368 16071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.712392 16071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.713611 16071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.713656 16071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.713683 16071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:11.715173 16071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:38543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=43549
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.715436 16071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.715698 16071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.720670 16108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.720679 16105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.720697 16106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.721158 16071 server_base.cc:1034] running on GCE node
I20260925 12:24:11.721354 16071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.721597 16071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.722782 16071 hybrid_clock.cc:648] HybridClock initialized: now 1790339051722762 us; error 36 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.725602 16071 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:11.726064 16071 webserver.cc:460] Webserver started at http://127.1.47.129:43549/ using document root <none> and password file <none>
I20260925 12:24:11.726310 16071 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.726379 16071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.727782 16071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.728600 16115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.728830 16071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.728950 16071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "3cd12663f94a48488454d2f2b31d0280"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "70de5d4cf23c285c3ad32c2ff4d5959a"
server_key_iv: "2f3c7e92cfa8ad7c79a0551294f4f76d"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.729303 16071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.744357 16071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.746803 16071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.746989 16071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.747247 16071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:11.747692 16122 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 12:24:11.749740 16071 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 12:24:11.749836 16071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 12:24:11.749881 16071 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 12:24:11.750564 16071 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 12:24:11.750626 16071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 12:24:11.750694 16122 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap starting.
I20260925 12:24:11.752625 16122 log.cc:824] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:11.755764 16122 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:11.756115 16122 tablet_bootstrap.cc:492] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Bootstrap complete.
I20260925 12:24:11.756275 16122 ts_tablet_manager.cc:1397] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260925 12:24:11.757735 16071 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:38543
I20260925 12:24:11.757771 16230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:38543 every 8 connection(s)
I20260925 12:24:11.758113 16071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:11.758081 16122 raft_consensus.cc:348] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.758631 16122 raft_consensus.cc:729] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd12663f94a48488454d2f2b31d0280, State: Initialized, Role: FOLLOWER
I20260925 12:24:11.758778 16122 consensus_queue.cc:260] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:11.759012 16122 ts_tablet_manager.cc:1428] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:11.762027 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16071
I20260925 12:24:11.762298 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:41503
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=45835
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:11.819694 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:24:11.758845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:39407)
0925 12:24:11.759141 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.759188 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:11.759340 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:11.819472 (+ 60132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
W20260925 12:24:11.819936 16231 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37349 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:37349: Client connection negotiation failed: client connection to 127.1.47.190:37349: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 12:24:11.820290 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.820166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:52493)
0925 12:24:11.820226 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.820270 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 12:24:11.820674 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.820588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:55149)
0925 12:24:11.820636 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.820658 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 12:24:11.820802 16231 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:24:11.844390 16235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.844597 16235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.844621 16235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.845757 16235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.845798 16235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.845851 16235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:11.847348 16235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:41503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=45835
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.847587 16235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.847815 16235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.853027 16243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.853098 16240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.853037 16241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.853569 16235 server_base.cc:1034] running on GCE node
I20260925 12:24:11.853788 16235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.854027 16235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.855170 16235 hybrid_clock.cc:648] HybridClock initialized: now 1790339051855148 us; error 38 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:11.857762 16235 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:11.858163 16235 webserver.cc:460] Webserver started at http://127.1.47.130:45835/ using document root <none> and password file <none>
I20260925 12:24:11.858342 16235 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:11.858384 16235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:11.859637 16235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:11.860385 16250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.860561 16235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.860648 16235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "91544717afc643afb794e15d1fe90bb4"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "3adbd47196bf3fa0727b743f7f27b681"
server_key_iv: "0ee1b293ba83328754664b6022bd883e"
server_key_version: "encryptionkey@0"
I20260925 12:24:11.860976 16235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:11.889446 16235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:11.892225 16235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:11.892356 16235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:11.892567 16235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:11.893126 16257 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260925 12:24:11.894567 16257 ts_tablet_manager.cc:1852] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260925 12:24:11.894860 16257 ts_tablet_manager.cc:1905] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:11.897064 16257 ts_tablet_manager.cc:1918] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:11.897168 16257 log.cc:1196] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:11.897446 16257 ts_tablet_manager.cc:1939] T dc51c7dd2a8a495aab6473635a119480 P 91544717afc643afb794e15d1fe90bb4: Deleting consensus metadata
I20260925 12:24:11.897673 16235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260925 12:24:11.897720 16235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260925 12:24:11.897747 16235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:11.897799 16235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:11.903800 16235 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:41503
I20260925 12:24:11.903874 16364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:41503 every 8 connection(s)
I20260925 12:24:11.904192 16235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:11.905243 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.904815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:45223)
0925 12:24:11.904984 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.905089 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
W20260925 12:24:11.905478 16365 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37349 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:37349: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
I20260925 12:24:11.905759 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.905678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:53319)
0925 12:24:11.905720 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.905745 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 12:24:11.906050 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:11.905978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:49245)
0925 12:24:11.906017 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:11.906037 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 12:24:11.906154 16365 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 12:24:11.907925 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16235
I20260925 12:24:11.908205 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:46845
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:37349
--webserver_port=37757
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:11.988991 16368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:11.989209 16368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:11.989233 16368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:11.990419 16368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:11.990468 16368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:11.990496 16368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:11.992012 16368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:46845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=37757
--tserver_master_addrs=127.1.47.190:37349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:11.992259 16368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:11.992504 16368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:11.997539 16376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.997517 16373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:11.997530 16374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:11.997792 16368 server_base.cc:1034] running on GCE node
I20260925 12:24:11.998095 16368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:11.998337 16368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:11.999511 16368 hybrid_clock.cc:648] HybridClock initialized: now 1790339051999501 us; error 30 us; skew 500 ppm
Sep 25 12:24:11 dist-test-slave-l9ww krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:12.001984 16368 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:12.002465 16368 webserver.cc:460] Webserver started at http://127.1.47.131:37757/ using document root <none> and password file <none>
I20260925 12:24:12.002715 16368 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:12.002761 16368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:12.004158 16368 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:24:12.004720 16383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:12.004911 16368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:24:12.004983 16368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "046a89c75cd5496986042b29ae07bba7"
format_stamp: "Formatted at 2026-09-25 12:24:10 on dist-test-slave-l9ww"
server_key: "1ed87b61f6f7bc4a1571912ce369fe75"
server_key_iv: "ba303bbee1f23c93709d4d14f46c7101"
server_key_version: "encryptionkey@0"
I20260925 12:24:12.005293 16368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:12.014148 16368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:12.015960 16368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:12.016069 16368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:12.016247 16368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:12.016497 16368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:12.016526 16368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:12.016548 16368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:12.016561 16368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:12.022231 16368 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:46845
I20260925 12:24:12.022280 16496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:46845 every 8 connection(s)
I20260925 12:24:12.022608 16368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:24:12.022969 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16368
I20260925 12:24:12.023640 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:12.023236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:56683)
0925 12:24:12.023398 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:12.023475 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
W20260925 12:24:12.024083 16497 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:37349 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:37349: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
I20260925 12:24:12.024358 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:12.024262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:54895)
0925 12:24:12.024307 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:12.024334 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 12:24:12.024680 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:12.024596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:34745)
0925 12:24:12.024639 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:12.024662 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 12:24:12.024834 16497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 12:24:12.821413 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:12.821160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:52437)
0925 12:24:12.821325 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:12.821363 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260925 12:24:12.906723 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:12.906520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:45443)
0925 12:24:12.906649 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:12.906687 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260925 12:24:13.025444 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.025159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:52799)
0925 12:24:13.025294 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:13.025335 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260925 12:24:13.608853 16504 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:13.608978 16504 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:13.609288 16504 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503), 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
Sep 25 12:24:13 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.130@KRBTEST.COM
Sep 25 12:24:13 dist-test-slave-l9ww krb5kdc[14930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339051, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.131@KRBTEST.COM
I20260925 12:24:13.620354 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.609506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:41503 (local address 127.1.47.129:47137)
0925 12:24:13.609621 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:13.609633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:13.609726 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:13.610622 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:13.610625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:13.611021 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:13.611235 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:13.611241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.612495 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.612497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:13.613517 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:13.613519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.614091 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.614092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:13.614217 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:13.614857 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:13.614861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:13.616357 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:13.618727 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:13.618733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:13.618735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:13.618988 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:13.619239 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:13.619240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:13.619351 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:13.619756 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:13.619762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:13.619943 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 12:24:13.620067 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 12:24:13.620253 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260925 12:24:13.620354 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.609659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:47137 (local address 127.1.47.130:41503)
0925 12:24:13.609840 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:13.609854 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:13.609881 (+ 27us) server_negotiation.cc:407] Connection header received
0925 12:24:13.610368 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:13.610371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:13.610414 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:13.610586 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:13.611377 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.612398 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.613601 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.613999 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.614114 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:13.616477 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:13.616481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:13.616481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:13.616493 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:13.618632 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619072 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:13.619074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:13.619075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:13.619161 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619423 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:13.619424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:13.619424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:13.619678 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:13.619774 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:13.620117 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 12:24:13.620239 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":57,"threads_started":1}
I20260925 12:24:13.620486 16508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.609803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:59759 (local address 127.1.47.131:46845)
0925 12:24:13.609940 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:13.609948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:13.609964 (+ 16us) server_negotiation.cc:407] Connection header received
0925 12:24:13.610373 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:13.610375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:13.610414 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:13.610586 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:13.611583 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.612483 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.613614 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.613998 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.614115 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:13.616808 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:13.616812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:13.616813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:13.616824 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:13.618632 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619299 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:13.619303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:13.619303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:13.619408 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619621 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:13.619622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:13.619623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:13.619898 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:13.619981 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:13.620182 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 12:24:13.620382 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260925 12:24:13.620481 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.609682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46845 (local address 127.1.47.129:59759)
0925 12:24:13.609763 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:13.609773 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:13.609824 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:13.610622 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:13.610625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:13.611021 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:13.611469 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:13.611472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.612588 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.612589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:13.613515 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:13.613517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:13.614090 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:13.614091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:13.614212 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:13.614800 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:13.614808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:13.616714 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:13.619071 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619071 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:13.619072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:13.619203 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:13.619483 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:13.619484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:13.619485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:13.619548 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:13.619973 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:13.619974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:13.620027 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 12:24:13.620119 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 12:24:13.620340 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":148,"thread_start_us":33,"threads_started":1}
W20260925 12:24:13.621482 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
W20260925 12:24:13.621562 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:13.621580 16118 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280; no voters: 046a89c75cd5496986042b29ae07bba7, 91544717afc643afb794e15d1fe90bb4
I20260925 12:24:13.621672 16504 raft_consensus.cc:2726] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:24:13.822163 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.821967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:48993)
0925 12:24:13.822065 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:13.822119 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 12:24:13.907584 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:13.907356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:40489)
0925 12:24:13.907489 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:13.907542 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260925 12:24:14.026130 16510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:14.025912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:47883)
0925 12:24:14.026033 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:14.026072 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260925 12:24:14.823017 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:14.822774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:49013)
0925 12:24:14.822917 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:14.822962 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260925 12:24:14.908396 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:14.908148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:43313)
0925 12:24:14.908277 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:14.908336 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260925 12:24:15.026881 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:15.026642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:38031)
0925 12:24:15.026797 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:15.026838 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260925 12:24:15.642652 16514 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:15.642804 16514 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:15.642954 16514 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503), 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
W20260925 12:24:15.643587 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
W20260925 12:24:15.643668 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:15.643720 16118 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280; no voters: 046a89c75cd5496986042b29ae07bba7, 91544717afc643afb794e15d1fe90bb4
I20260925 12:24:15.643810 16514 raft_consensus.cc:2726] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:24:15.823957 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:15.823661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:38647)
0925 12:24:15.823812 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:15.823895 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260925 12:24:15.909173 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:15.908954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:47601)
0925 12:24:15.909084 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:15.909128 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260925 12:24:16.027592 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:16.027364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:45765)
0925 12:24:16.027500 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:16.027538 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260925 12:24:16.824803 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:16.824550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:42451)
0925 12:24:16.824711 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:16.824753 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260925 12:24:16.909960 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:16.909692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:48427)
0925 12:24:16.909867 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:16.909910 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260925 12:24:17.028430 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:17.028137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:59245)
0925 12:24:17.028288 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:17.028377 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260925 12:24:17.285526 16521 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:17.285651 16521 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:17.285825 16521 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503), 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
W20260925 12:24:17.286188 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
W20260925 12:24:17.286262 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:17.286278 16118 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280; no voters: 046a89c75cd5496986042b29ae07bba7, 91544717afc643afb794e15d1fe90bb4
I20260925 12:24:17.286351 16521 raft_consensus.cc:2726] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:24:17.825681 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:17.825424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:38215)
0925 12:24:17.825582 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:17.825629 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
I20260925 12:24:17.910790 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:17.910559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:47441)
0925 12:24:17.910702 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:17.910747 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
I20260925 12:24:18.029289 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:18.029027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:56977)
0925 12:24:18.029182 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:18.029225 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
I20260925 12:24:18.826509 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:18.826242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:39191)
0925 12:24:18.826414 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:18.826459 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260925 12:24:18.905117 16526 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:18.905205 16526 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:18.905360 16526 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503), 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
W20260925 12:24:18.905719 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
W20260925 12:24:18.905813 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:18.905830 16118 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280; no voters: 046a89c75cd5496986042b29ae07bba7, 91544717afc643afb794e15d1fe90bb4
I20260925 12:24:18.905902 16526 raft_consensus.cc:2726] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:24:18.911511 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:18.911309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:51267)
0925 12:24:18.911431 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:18.911473 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260925 12:24:19.030161 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:19.029912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:33379)
0925 12:24:19.030074 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:19.030114 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":119,"threads_started":1}
I20260925 12:24:19.827390 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:19.827161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:40203)
0925 12:24:19.827302 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:19.827342 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260925 12:24:19.912231 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:19.911997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:52465)
0925 12:24:19.912136 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:19.912177 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260925 12:24:20.030966 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:20.030745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:35031)
0925 12:24:20.030878 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:20.030920 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260925 12:24:20.828339 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:20.828086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:49449)
0925 12:24:20.828237 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:20.828283 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":60,"threads_started":1}
I20260925 12:24:20.912912 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:20.912709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:43299)
0925 12:24:20.912828 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:20.912869 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260925 12:24:21.031659 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:21.031408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:45447)
0925 12:24:21.031561 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:21.031605 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":58,"threads_started":1}
I20260925 12:24:21.829101 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:21.828889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:39451)
0925 12:24:21.829016 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:21.829052 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260925 12:24:21.913668 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:21.913441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:59201)
0925 12:24:21.913579 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:21.913621 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260925 12:24:22.032405 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:22.032176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:49731)
0925 12:24:22.032317 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:22.032358 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260925 12:24:22.829984 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:22.829669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:57003)
0925 12:24:22.829873 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:22.829931 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260925 12:24:22.914507 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:22.914256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:53383)
0925 12:24:22.914416 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:22.914461 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260925 12:24:23.033300 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:23.033038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:52753)
0925 12:24:23.033207 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:23.033252 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":64,"threads_started":1}
I20260925 12:24:23.830883 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:23.830648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:54699)
0925 12:24:23.830797 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:23.830828 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260925 12:24:23.915310 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:23.915060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:39639)
0925 12:24:23.915223 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:23.915262 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260925 12:24:24.034099 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:24.033855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:40635)
0925 12:24:24.034012 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:24.034049 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260925 12:24:24.831697 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:24.831469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:45299)
0925 12:24:24.831592 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:24.831634 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260925 12:24:24.916162 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:24.915956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:55393)
0925 12:24:24.916081 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:24.916117 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260925 12:24:25.034878 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:25.034600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:33585)
0925 12:24:25.034760 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:25.034805 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260925 12:24:25.832461 16547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:25.832241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:56521)
0925 12:24:25.832373 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:25.832413 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260925 12:24:25.916986 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:25.916722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:57207)
0925 12:24:25.916882 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:25.916926 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260925 12:24:26.035748 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:26.035497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.131:44989)
0925 12:24:26.035652 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:26.035697 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260925 12:24:26.419819 16550 raft_consensus.cc:482] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:26.420001 16550 raft_consensus.cc:504] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91544717afc643afb794e15d1fe90bb4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 41503 } } peers { permanent_uuid: "046a89c75cd5496986042b29ae07bba7" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46845 } } peers { permanent_uuid: "3cd12663f94a48488454d2f2b31d0280" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 38543 } }
I20260925 12:24:26.420202 16550 leader_election.cc:288] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503), 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845)
W20260925 12:24:26.420580 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91544717afc643afb794e15d1fe90bb4 (127.1.47.130:41503): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
W20260925 12:24:26.420686 16118 leader_election.cc:341] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 046a89c75cd5496986042b29ae07bba7 (127.1.47.131:46845): Not found: Tablet not found: dc51c7dd2a8a495aab6473635a119480
I20260925 12:24:26.420714 16118 leader_election.cc:302] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cd12663f94a48488454d2f2b31d0280; no voters: 046a89c75cd5496986042b29ae07bba7, 91544717afc643afb794e15d1fe90bb4
I20260925 12:24:26.420797 16550 raft_consensus.cc:2726] T dc51c7dd2a8a495aab6473635a119480 P 3cd12663f94a48488454d2f2b31d0280 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:24:26.833305 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:26.833045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.129:57649)
0925 12:24:26.833202 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:26.833248 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260925 12:24:26.917936 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:26.917683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37349 (local address 127.1.47.130:45103)
0925 12:24:26.917841 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:26.917885 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:37349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260925 12:24:27.025288 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16071
I20260925 12:24:27.030293 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16235
I20260925 12:24:27.035032 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16368
2026-09-25T12:24:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18282 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:24:27 dist-test-slave-l9ww krb5kdc[16555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:24:27 dist-test-slave-l9ww krb5kdc[16555](info): set up 2 sockets
Sep 25 12:24:27 dist-test-slave-l9ww krb5kdc[16555](info): commencing operation
krb5kdc: starting...
W20260925 12:24:28.080024 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:24:28.204753 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:28.285480 16568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:28.285737 16568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:28.285771 16568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:28.286909 16568 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 12:24:28.286980 16568 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 12:24:28.287003 16568 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 12:24:28.287034 16568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:28.287063 16568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:28.287094 16568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:28.287122 16568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:28.287145 16568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:28.288686 16568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:28.288966 16568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:28.289192 16568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:28.294055 16573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.294279 16574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.294207 16568 server_base.cc:1034] running on GCE node
W20260925 12:24:28.294059 16576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.294615 16568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:28.294912 16568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:28.296087 16568 hybrid_clock.cc:648] HybridClock initialized: now 1790339068296079 us; error 35 us; skew 500 ppm
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:28.298664 16568 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:28.299124 16568 webserver.cc:460] Webserver started at http://127.1.47.190:33699/ using document root <none> and password file <none>
I20260925 12:24:28.299422 16568 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:28.299486 16568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:28.299580 16568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:28.300657 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "48c9965cf75e48798325e71fa8270b04"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "4bd38bc98f09cb7cf39c57edd25f739d"
server_key_iv: "14ea292611c7a054a11a34755a882f64"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.300992 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "48c9965cf75e48798325e71fa8270b04"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "4bd38bc98f09cb7cf39c57edd25f739d"
server_key_iv: "14ea292611c7a054a11a34755a882f64"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.302273 16568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:24:28.303042 16583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.303254 16568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:28.303386 16568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "48c9965cf75e48798325e71fa8270b04"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "4bd38bc98f09cb7cf39c57edd25f739d"
server_key_iv: "14ea292611c7a054a11a34755a882f64"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.303452 16568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:28.321357 16568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:28.323601 16568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:28.323733 16568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:28.327937 16568 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:38531
I20260925 12:24:28.327993 16635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:38531 every 8 connection(s)
I20260925 12:24:28.328359 16568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:28.329264 16636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:28.329864 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16568
I20260925 12:24:28.329962 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:24:28.330176 1214 external_mini_cluster.cc:1428] Setting key 61f9a1e3a523e156d9b67dc7f87559b7
I20260925 12:24:28.332440 16636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap starting.
I20260925 12:24:28.333039 16636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:28.333308 16636 log.cc:824] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:28.333977 16636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: No bootstrap required, opened a new log
I20260925 12:24:28.336076 16636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:28.336241 16636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:28.336306 16636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Initialized, Role: FOLLOWER
I20260925 12:24:28.336463 16636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:28.336560 16636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:28.336606 16636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:28.336653 16636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:28.337328 16636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:28.337447 16636 leader_election.cc:302] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c9965cf75e48798325e71fa8270b04; no voters:
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:28.337620 16636 leader_election.cc:288] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:24:28.337715 16641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:28.337922 16641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 LEADER]: Becoming Leader. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Running, Role: LEADER
I20260925 12:24:28.337950 16636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:28.338057 16641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:28.338492 16641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c9965cf75e48798325e71fa8270b04. Latest consensus state: current_term: 1 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:24:28.338660 16641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:28.338543 16642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:24:28.339234 16642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
W20260925 12:24:28.339761 16656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:24:28.339828 16656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:24:28.339946 16650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:28.340236 16650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:28.341703 16650 catalog_manager.cc:1347] Generated new cluster ID: 638f226b37ed4b80a4abcd0b9e849fa5
I20260925 12:24:28.341753 16650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:28.342840 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.330730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.1.47.190:38531)
0925 12:24:28.330915 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.330924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.330950 (+ 26us) server_negotiation.cc:407] Connection header received
0925 12:24:28.331445 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.331453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.331504 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.331624 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:28.332453 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.333592 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.334511 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.334984 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.335141 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.338196 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:28.338200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:28.338200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:28.338214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:28.340575 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.341035 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.341038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.341038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.341145 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.341418 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.341419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.341419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.341750 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:28.341895 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.342469 (+ 574us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.342620 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260925 12:24:28.357409 16650 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:24:28.358035 16650 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:28.361161 16650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Generated new TSK 0
I20260925 12:24:28.361367 16650 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:24:28.391124 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:28.478655 16662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:28.478880 16662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:28.478905 16662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:28.480165 16662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:28.480211 16662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:28.480239 16662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:28.481768 16662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:28.482029 16662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:28.482286 16662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:28.487385 16670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.487392 16667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.487455 16668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.487722 16662 server_base.cc:1034] running on GCE node
I20260925 12:24:28.488003 16662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:28.488210 16662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:28.489399 16662 hybrid_clock.cc:648] HybridClock initialized: now 1790339068489404 us; error 53 us; skew 500 ppm
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:28.492260 16662 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:28.492802 16662 webserver.cc:460] Webserver started at http://127.1.47.129:34999/ using document root <none> and password file <none>
I20260925 12:24:28.493045 16662 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:28.493113 16662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:28.493271 16662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:28.494374 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "a28eb6075671fee618b85485a6a39ba4"
server_key_iv: "f0ecbb08b26905297ca09c01f3031052"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.494755 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "a28eb6075671fee618b85485a6a39ba4"
server_key_iv: "f0ecbb08b26905297ca09c01f3031052"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.496227 16662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:24:28.497045 16677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.497224 16662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:28.497328 16662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "a28eb6075671fee618b85485a6a39ba4"
server_key_iv: "f0ecbb08b26905297ca09c01f3031052"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.497406 16662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:28.520552 16662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:28.522830 16662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:28.523015 16662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:28.523259 16662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:28.523644 16662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:28.523705 16662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.523741 16662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:28.523767 16662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.529839 16662 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43891
I20260925 12:24:28.529883 16790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43891 every 8 connection(s)
I20260925 12:24:28.530226 16662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:28.536408 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16662
I20260925 12:24:28.536525 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:24:28.536723 1214 external_mini_cluster.cc:1428] Setting key 88a49c2d7c5bd4cc32927eaf8c89b18e
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:28.540643 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.531032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:43637 (local address 127.1.47.190:38531)
0925 12:24:28.531081 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.531083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.531304 (+ 221us) server_negotiation.cc:407] Connection header received
0925 12:24:28.531845 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.531847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.531886 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.531940 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:28.532926 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.533446 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.534465 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.534671 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.534760 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.537331 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:28.537333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:28.537333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:28.537340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:28.539086 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.539580 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.539582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.539582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.539644 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.539952 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.539953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.539953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.540071 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:28.540143 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.540486 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.540572 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:24:28.540920 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.530991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:43637)
0925 12:24:28.531180 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.531218 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.531736 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.532030 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.532037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.532624 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:28.532821 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.532826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.533545 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.533546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.534365 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.534368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.534774 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.534775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.534902 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.535430 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:28.535440 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:28.537224 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:28.539179 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.539186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.539188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.539426 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.539712 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.539713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.539713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.539834 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.540144 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:28.540152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:28.540326 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.540451 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.540637 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260925 12:24:28.541515 16791 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:28.541641 16791 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:28.541826 16791 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:28.542411 16600 ts_manager.cc:194] Registered new tserver with Master: ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891)
I20260925 12:24:28.543042 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:43637
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:24:28.563427 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:28.646309 16796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:28.646540 16796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:28.646565 16796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:28.647679 16796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:28.647715 16796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:28.647742 16796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:28.649255 16796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:28.649457 16796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:28.649632 16796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:28.655094 16802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.655256 16796 server_base.cc:1034] running on GCE node
W20260925 12:24:28.655112 16801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.655423 16804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.655661 16796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:28.655992 16796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:28.657169 16796 hybrid_clock.cc:648] HybridClock initialized: now 1790339068657149 us; error 40 us; skew 500 ppm
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:28.659768 16796 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:28.660219 16796 webserver.cc:460] Webserver started at http://127.1.47.130:44019/ using document root <none> and password file <none>
I20260925 12:24:28.660457 16796 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:28.660527 16796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:28.660681 16796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:28.661747 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "9b9f27d7afea97306047b1ad9bac7818"
server_key_iv: "3f7af2db7125d9d54392378f53741a7d"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.662132 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "9b9f27d7afea97306047b1ad9bac7818"
server_key_iv: "3f7af2db7125d9d54392378f53741a7d"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.663568 16796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:24:28.664551 16811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.664774 16796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:24:28.664908 16796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "9b9f27d7afea97306047b1ad9bac7818"
server_key_iv: "3f7af2db7125d9d54392378f53741a7d"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.664996 16796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:28.691471 16796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:28.694221 16796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:28.694418 16796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:28.694705 16796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:28.695075 16796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:28.695143 16796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.695194 16796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:28.695222 16796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.702183 16796 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:40225
I20260925 12:24:28.702231 16924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:40225 every 8 connection(s)
I20260925 12:24:28.702605 16796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:28.708926 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16796
I20260925 12:24:28.709031 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:24:28.709215 1214 external_mini_cluster.cc:1428] Setting key b1b50dfd85c0bd1a4a6d9b87b1865232
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:28.713861 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.703615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:57373 (local address 127.1.47.190:38531)
0925 12:24:28.703709 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.703711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.703788 (+ 77us) server_negotiation.cc:407] Connection header received
0925 12:24:28.704384 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.704385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.704399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.704448 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:28.705582 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.706142 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.707297 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.707518 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.707615 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.710281 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:28.710283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:28.710284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:28.710292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:28.712213 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.712718 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.712719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.712719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.712784 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.713073 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.713074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.713074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.713232 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:28.713304 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.713625 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.713722 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 12:24:28.713977 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.703403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:57373)
0925 12:24:28.703600 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.703676 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.704252 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.704531 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.704535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.705178 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:28.705448 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.705453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.706270 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.706271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.707191 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.707194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.707625 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.707626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.707762 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.708378 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:28.708387 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:28.710180 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:28.712335 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.712342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.712345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.712630 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.712866 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.712867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.712867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.712996 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.713312 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:28.713320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:28.713472 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.713574 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.713719 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260925 12:24:28.714480 16925 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:28.714601 16925 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:28.714818 16925 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:28.715332 16600 ts_manager.cc:194] Registered new tserver with Master: a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225)
I20260925 12:24:28.715790 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:57373
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:24:28.739760 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:28.822513 16930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:28.822786 16930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:28.822827 16930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:28.824069 16930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:28.824153 16930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:28.824196 16930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:28.825738 16930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:28.826023 16930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:28.826294 16930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:28.831960 16936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.831945 16935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:28.831981 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:28.832250 16930 server_base.cc:1034] running on GCE node
I20260925 12:24:28.832464 16930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:28.832695 16930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:28.833920 16930 hybrid_clock.cc:648] HybridClock initialized: now 1790339068833890 us; error 41 us; skew 500 ppm
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:28.836777 16930 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:28.837201 16930 webserver.cc:460] Webserver started at http://127.1.47.131:39493/ using document root <none> and password file <none>
I20260925 12:24:28.837399 16930 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:28.837443 16930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:28.837574 16930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:24:28.838815 16930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "81c8f02e529b49999f2b60824466fcf4"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "0fe73f6156eb7d4344c10373b41fc562"
server_key_iv: "7334633707babfdc732637c4e48fad69"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.839200 16930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "81c8f02e529b49999f2b60824466fcf4"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "0fe73f6156eb7d4344c10373b41fc562"
server_key_iv: "7334633707babfdc732637c4e48fad69"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.840772 16930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:24:28.841636 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.841809 16930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:28.841910 16930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "81c8f02e529b49999f2b60824466fcf4"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "0fe73f6156eb7d4344c10373b41fc562"
server_key_iv: "7334633707babfdc732637c4e48fad69"
server_key_version: "encryptionkey@0"
I20260925 12:24:28.842000 16930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:28.866088 16930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:28.868323 16930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:28.868507 16930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:28.868745 16930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:28.869064 16930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:28.869119 16930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.869160 16930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:28.869187 16930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:28.875077 16930 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38369
I20260925 12:24:28.875121 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38369 every 8 connection(s)
I20260925 12:24:28.875512 16930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:24:28 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:28.884969 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.876338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:60455 (local address 127.1.47.190:38531)
0925 12:24:28.876393 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.876396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.876625 (+ 229us) server_negotiation.cc:407] Connection header received
0925 12:24:28.877181 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.877183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.877197 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.877244 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:28.878222 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.878733 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.879758 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.880021 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.880107 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.882400 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:28.882401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:28.882402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:28.882410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:28.883696 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.884144 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.884145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.884145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.884197 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.884412 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.884413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.884413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.884526 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:28.884576 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.884837 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.884899 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:24:28.885130 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.876320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:60455)
0925 12:24:28.876494 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.876534 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.877063 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.877327 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.877330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.877912 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:28.878119 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.878124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.878837 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.878838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.879657 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.879659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.880116 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.880118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.880246 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.880812 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:28.880821 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:28.882251 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:28.883777 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.883783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.883785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.884078 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.884260 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:28.884260 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:28.884261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:28.884359 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:28.884582 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:28.884593 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:28.884714 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.884801 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.884925 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260925 12:24:28.885522 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 16930
I20260925 12:24:28.885605 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:24:28.885632 17059 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:28.885733 17059 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:28.885799 1214 external_mini_cluster.cc:1428] Setting key 25cd154b7cc157696eeb29599e35ef48
I20260925 12:24:28.885926 17059 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:28.886350 16600 ts_manager.cc:194] Registered new tserver with Master: 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
I20260925 12:24:28.886729 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:28.886862 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:60455
I20260925 12:24:28.894296 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.888377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.1.47.190:38531)
0925 12:24:28.888463 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.888465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.888538 (+ 73us) server_negotiation.cc:407] Connection header received
0925 12:24:28.888612 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.888613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.888622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.888677 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:28.889231 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.889711 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.890468 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.890677 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.890762 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.891877 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:28.891879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:28.891879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:28.891886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:28.893111 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.893455 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.893456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.893456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.893533 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:28.893731 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:28.893732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:28.893733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:28.893889 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:28.893999 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.894110 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.894195 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 12:24:28.896046 16600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:24:28.897549 16600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:24:28.904418 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.899935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.1.47.130:40225)
0925 12:24:28.900381 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.900386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.900394 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:28.900436 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.900438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.900450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.900543 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:28.900707 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.901852 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.903216 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.903990 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904069 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904130 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.904284 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.904328 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":98,"threads_started":1}
I20260925 12:24:28.904405 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.899810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40225 (local address 127.0.0.1:40046)
0925 12:24:28.900135 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.900169 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.900231 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.900546 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.900549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.900551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:28.900629 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.900630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.901956 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.901957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.903119 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.903121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.904079 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.904163 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904177 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.904253 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.904315 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":69,"thread_start_us":108,"threads_started":1}
I20260925 12:24:28.904596 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.900107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.1.47.131:38369)
0925 12:24:28.900294 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.900300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.900315 (+ 15us) server_negotiation.cc:407] Connection header received
0925 12:24:28.900682 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.900685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.900701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.900771 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:28.901150 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.901880 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.903476 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.904060 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904126 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904184 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.904454 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.904502 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260925 12:24:28.904418 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.899982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38369 (local address 127.0.0.1:44610)
0925 12:24:28.900269 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.900279 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.900322 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.900918 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.900919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.900921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:28.901029 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.901031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.902011 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.902012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.903373 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.903375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.904215 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.904282 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904286 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.904330 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.904362 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":98,"thread_start_us":57,"threads_started":1}
I20260925 12:24:28.905125 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.899747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.0.0.1:51704)
0925 12:24:28.900104 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.900149 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.900255 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.900621 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.900622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.900623 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:28.900672 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.900674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.902189 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.902190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.903374 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.903375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.904571 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.904954 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904959 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.905010 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.905057 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":12,"thread_start_us":66,"threads_started":1}
I20260925 12:24:28.905243 16859 tablet_service.cc:1467] Processing CreateTablet for tablet e51213e4d9d142a5886d8ec68067f7df (DEFAULT_TABLE table=test-table [id=db159cdbb65f4fe792025a2d4511bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:28.905390 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.899917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.1.47.129:43891)
0925 12:24:28.900389 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.900393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.900402 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:24:28.900449 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.900451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.900468 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.900541 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:28.900738 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.901846 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.903471 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.904187 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.904286 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.904349 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.905280 (+ 931us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.905335 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":92,"threads_started":1}
I20260925 12:24:28.905536 16993 tablet_service.cc:1467] Processing CreateTablet for tablet e51213e4d9d142a5886d8ec68067f7df (DEFAULT_TABLE table=test-table [id=db159cdbb65f4fe792025a2d4511bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:28.905620 16859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51213e4d9d142a5886d8ec68067f7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:28.905859 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51213e4d9d142a5886d8ec68067f7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:28.905983 16725 tablet_service.cc:1467] Processing CreateTablet for tablet e51213e4d9d142a5886d8ec68067f7df (DEFAULT_TABLE table=test-table [id=db159cdbb65f4fe792025a2d4511bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:28.906226 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51213e4d9d142a5886d8ec68067f7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:28.908949 17076 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap starting.
I20260925 12:24:28.909301 17078 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Bootstrap starting.
I20260925 12:24:28.909634 17076 tablet_bootstrap.cc:654] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:28.909937 17078 tablet_bootstrap.cc:654] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:28.910074 17076 log.cc:824] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:28.910521 17078 log.cc:824] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:28.910786 17076 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: No bootstrap required, opened a new log
I20260925 12:24:28.910882 17077 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Bootstrap starting.
I20260925 12:24:28.910943 17076 ts_tablet_manager.cc:1397] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:28.911522 17077 tablet_bootstrap.cc:654] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:28.912081 17077 log.cc:824] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:28.912245 17078 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: No bootstrap required, opened a new log
I20260925 12:24:28.912350 17078 ts_tablet_manager.cc:1397] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:24:28.912861 17077 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: No bootstrap required, opened a new log
I20260925 12:24:28.912791 17076 raft_consensus.cc:348] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.912928 17077 ts_tablet_manager.cc:1397] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:24:28.912949 17076 raft_consensus.cc:374] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:28.913023 17076 raft_consensus.cc:729] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab337d38bf8d43cc85aa2a703d07e67e, State: Initialized, Role: FOLLOWER
I20260925 12:24:28.913177 17076 consensus_queue.cc:260] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.913453 17076 ts_tablet_manager.cc:1428] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:28.913590 16791 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:28.914336 17078 raft_consensus.cc:348] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.914541 17078 raft_consensus.cc:374] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:28.914597 17078 raft_consensus.cc:729] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c8f02e529b49999f2b60824466fcf4, State: Initialized, Role: FOLLOWER
I20260925 12:24:28.914703 17078 consensus_queue.cc:260] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.914858 17077 raft_consensus.cc:348] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.914994 17059 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:28.915004 17078 ts_tablet_manager.cc:1428] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:28.915030 17077 raft_consensus.cc:374] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:28.915089 17077 raft_consensus.cc:729] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a555ccab69e34ae2a9ef04651ff7db0e, State: Initialized, Role: FOLLOWER
I20260925 12:24:28.915228 17077 consensus_queue.cc:260] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.915512 17077 ts_tablet_manager.cc:1428] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:24:28.915620 16925 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:28.945102 17083 raft_consensus.cc:482] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:28.945253 17083 raft_consensus.cc:504] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.945693 17083 leader_election.cc:288] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891), a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225)
I20260925 12:24:28.948588 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.945962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:55759 (local address 127.1.47.129:43891)
0925 12:24:28.946010 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.946012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.946022 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:24:28.946090 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.946091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.946103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.946148 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:28.946318 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.946797 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.947761 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.948219 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.948305 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.948314 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.948481 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.948520 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:24:28.948589 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.945979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:40609 (local address 127.1.47.130:40225)
0925 12:24:28.946049 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.946051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.946141 (+ 90us) server_negotiation.cc:407] Connection header received
0925 12:24:28.946205 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.946206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.946218 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.946267 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:28.946427 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.946898 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.947847 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.948235 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.948322 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.948330 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.948481 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.948522 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 12:24:28.948596 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.945864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.1.47.131:55759)
0925 12:24:28.945958 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.945973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.946027 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.946161 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.946162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.946163 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:28.946240 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.946242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.946884 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.946885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.947664 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.947667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.948302 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.948303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.948384 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.948387 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.948467 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.948512 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":33}
I20260925 12:24:28.948611 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.945864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40225 (local address 127.1.47.131:40609)
0925 12:24:28.946073 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.946089 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:28.946150 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:28.946269 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:28.946270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:28.946271 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:28.946343 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.946344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.946988 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.946989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.947753 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:28.947755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.948321 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.948322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:28.948394 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.948398 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:28.948474 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 12:24:28.948516 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":23,"thread_start_us":110,"threads_started":1}
I20260925 12:24:28.948993 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51213e4d9d142a5886d8ec68067f7df" candidate_uuid: "81c8f02e529b49999f2b60824466fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" is_pre_election: true
I20260925 12:24:28.949012 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51213e4d9d142a5886d8ec68067f7df" candidate_uuid: "81c8f02e529b49999f2b60824466fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" is_pre_election: true
I20260925 12:24:28.949167 16879 raft_consensus.cc:2454] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81c8f02e529b49999f2b60824466fcf4 in term 0.
I20260925 12:24:28.949168 16745 raft_consensus.cc:2454] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81c8f02e529b49999f2b60824466fcf4 in term 0.
I20260925 12:24:28.949388 16948 leader_election.cc:302] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c8f02e529b49999f2b60824466fcf4, ab337d38bf8d43cc85aa2a703d07e67e; no voters:
I20260925 12:24:28.949522 17083 raft_consensus.cc:2781] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:28.949580 17083 raft_consensus.cc:482] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:28.949604 17083 raft_consensus.cc:3037] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:28.950299 17083 raft_consensus.cc:504] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.950439 17083 leader_election.cc:288] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [CANDIDATE]: Term 1 election: Requested vote from peers ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891), a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225)
I20260925 12:24:28.950577 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51213e4d9d142a5886d8ec68067f7df" candidate_uuid: "81c8f02e529b49999f2b60824466fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
I20260925 12:24:28.950594 16879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51213e4d9d142a5886d8ec68067f7df" candidate_uuid: "81c8f02e529b49999f2b60824466fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
I20260925 12:24:28.950673 16879 raft_consensus.cc:3037] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:28.950678 16745 raft_consensus.cc:3037] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:28.951370 16745 raft_consensus.cc:2454] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81c8f02e529b49999f2b60824466fcf4 in term 1.
I20260925 12:24:28.951397 16879 raft_consensus.cc:2454] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81c8f02e529b49999f2b60824466fcf4 in term 1.
I20260925 12:24:28.951536 16948 leader_election.cc:302] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c8f02e529b49999f2b60824466fcf4, ab337d38bf8d43cc85aa2a703d07e67e; no voters:
I20260925 12:24:28.951644 17083 raft_consensus.cc:2781] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:28.951797 17083 raft_consensus.cc:686] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 1 LEADER]: Becoming Leader. State: Replica: 81c8f02e529b49999f2b60824466fcf4, State: Running, Role: LEADER
I20260925 12:24:28.952020 17083 consensus_queue.cc:237] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:28.952911 16600 catalog_manager.cc:5495] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "81c8f02e529b49999f2b60824466fcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } health_report { overall_health: UNKNOWN } } }
W20260925 12:24:28.953212 16926 tablet.cc:2369] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:24:28.961853 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.959403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.1.47.131:38369)
0925 12:24:28.959474 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:28.959476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:28.959484 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:28.959523 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:28.959524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:28.959544 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:28.959605 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:28.959763 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.960388 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.961150 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:28.961381 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:28.961453 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:28.961577 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:28.961685 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:28.961751 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:28.961769 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 12:24:28.961785 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:24:28.963645 16745 raft_consensus.cc:1261] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Refusing update from remote peer 81c8f02e529b49999f2b60824466fcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:28.963661 16879 raft_consensus.cc:1261] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Refusing update from remote peer 81c8f02e529b49999f2b60824466fcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:28.963990 17086 consensus_queue.cc:1035] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:28.964136 17083 consensus_queue.cc:1035] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:28.966507 17089 mvcc.cc:205] Tried to move back new op lower bound from 7333228826472558592 to 7333228826428194816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228826472558592}]
I20260925 12:24:28.967092 17090 mvcc.cc:205] Tried to move back new op lower bound from 7333228826472558592 to 7333228826428194816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228826472558592}]
I20260925 12:24:28.967295 17091 mvcc.cc:205] Tried to move back new op lower bound from 7333228826472558592 to 7333228826428194816. Current Snapshot: MvccSnapshot[applied={T|T < 7333228826472558592}]
I20260925 12:24:28.969774 16600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:28.969875 16600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:28.970652 16600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Sending DeleteTablet for 3 replicas of tablet e51213e4d9d142a5886d8ec68067f7df
I20260925 12:24:28.971010 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet e51213e4d9d142a5886d8ec68067f7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:28 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:51704
I20260925 12:24:28.971014 16859 tablet_service.cc:1514] Processing DeleteTablet for tablet e51213e4d9d142a5886d8ec68067f7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:28 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40046
I20260925 12:24:28.971024 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet e51213e4d9d142a5886d8ec68067f7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:28 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:44610
I20260925 12:24:28.971251 17100 tablet_replica.cc:330] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: stopping tablet replica
I20260925 12:24:28.971355 17101 tablet_replica.cc:330] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: stopping tablet replica
I20260925 12:24:28.971380 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16662
I20260925 12:24:28.971405 17100 raft_consensus.cc:2229] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:24:28.971508 17101 raft_consensus.cc:2229] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:28.971563 17100 raft_consensus.cc:2258] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:28.971612 17101 raft_consensus.cc:2258] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:28.971988 17099 tablet_replica.cc:330] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: stopping tablet replica
I20260925 12:24:28.972084 17099 raft_consensus.cc:2229] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:28.972178 17099 raft_consensus.cc:2258] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:28.972273 17100 ts_tablet_manager.cc:1905] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:28.972612 17099 ts_tablet_manager.cc:1905] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:28.973970 17100 ts_tablet_manager.cc:1918] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:28.974035 17100 log.cc:1196] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/e51213e4d9d142a5886d8ec68067f7df
I20260925 12:24:28.974293 17100 ts_tablet_manager.cc:1939] T e51213e4d9d142a5886d8ec68067f7df P 81c8f02e529b49999f2b60824466fcf4: Deleting consensus metadata
I20260925 12:24:28.974547 17099 ts_tablet_manager.cc:1918] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:28.974610 17099 log.cc:1196] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/e51213e4d9d142a5886d8ec68067f7df
I20260925 12:24:28.974880 17099 ts_tablet_manager.cc:1939] T e51213e4d9d142a5886d8ec68067f7df P a555ccab69e34ae2a9ef04651ff7db0e: Deleting consensus metadata
I20260925 12:24:28.974958 16586 catalog_manager.cc:4841] TS 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): tablet e51213e4d9d142a5886d8ec68067f7df (table test-table [id=db159cdbb65f4fe792025a2d4511bd76]) successfully deleted
I20260925 12:24:28.975291 16584 catalog_manager.cc:4841] TS a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): tablet e51213e4d9d142a5886d8ec68067f7df (table test-table [id=db159cdbb65f4fe792025a2d4511bd76]) successfully deleted
W20260925 12:24:28.978397 16586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43891 (error 108)
I20260925 12:24:28.978606 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16796
I20260925 12:24:28.979452 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:28.979306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.0.0.1:51714)
0925 12:24:28.979374 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:28.979416 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 12:24:28.979588 16586 catalog_manager.cc:4584] TS ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e51213e4d9d142a5886d8ec68067f7df: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43891: connect: Connection refused (error 111)
I20260925 12:24:28.985204 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16930
I20260925 12:24:28.989624 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 16568
I20260925 12:24:48.995417 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--webserver_interface=127.1.47.190
--webserver_port=33699
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:24:49.076069 17103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:49.076279 17103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:49.076303 17103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:49.077533 17103 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 12:24:49.077580 17103 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 12:24:49.077593 17103 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 12:24:49.077605 17103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:24:49.077617 17103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:49.077630 17103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:49.077675 17103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:24:49.077705 17103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:24:49.079183 17103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:49.079432 17103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:49.079669 17103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:49.088500 17110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.088567 17109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:49.088521 17103 server_base.cc:1034] running on GCE node
W20260925 12:24:49.088515 17112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:49.089023 17103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:49.089293 17103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:49.090471 17103 hybrid_clock.cc:648] HybridClock initialized: now 1790339089090449 us; error 36 us; skew 500 ppm
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:49.093222 17103 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:24:49.093680 17103 webserver.cc:460] Webserver started at http://127.1.47.190:33699/ using document root <none> and password file <none>
I20260925 12:24:49.093923 17103 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:49.094003 17103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:49.095391 17103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.096243 17119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.096509 17103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.096617 17103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "48c9965cf75e48798325e71fa8270b04"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "4bd38bc98f09cb7cf39c57edd25f739d"
server_key_iv: "14ea292611c7a054a11a34755a882f64"
server_key_version: "encryptionkey@0"
I20260925 12:24:49.097044 17103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:49.107703 17103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:49.109997 17103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:49.110113 17103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:49.113896 17103 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:38531
I20260925 12:24:49.113945 17171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:38531 every 8 connection(s)
I20260925 12:24:49.114326 17103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:24:49.115837 17172 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:24:49.116572 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap starting.
I20260925 12:24:49.120592 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17103
I20260925 12:24:49.120949 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:43891
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=34999
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:49.125583 17172 log.cc:824] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:49.128774 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:49.129184 17172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap complete.
I20260925 12:24:49.133013 17172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:49.133344 17172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Initialized, Role: FOLLOWER
I20260925 12:24:49.133512 17172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:49.133606 17172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:24:49.133661 17172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:24:49.133713 17172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 1 FOLLOWER]: Advancing to term 2
I20260925 12:24:49.135118 17172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:49.135234 17172 leader_election.cc:302] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c9965cf75e48798325e71fa8270b04; no voters:
I20260925 12:24:49.135419 17172 leader_election.cc:288] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 12:24:49.135546 17176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 12:24:49.135732 17172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:24:49.135788 17176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 LEADER]: Becoming Leader. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Running, Role: LEADER
I20260925 12:24:49.135993 17176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:24:49.136293 17177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:24:49.136401 17177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:49.136693 17178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c9965cf75e48798325e71fa8270b04. Latest consensus state: current_term: 2 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:24:49.136763 17178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:24:49.137005 17186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:24:49.137895 17186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:24:49.138016 17186 catalog_manager.cc:1259] Loaded cluster ID: 638f226b37ed4b80a4abcd0b9e849fa5
I20260925 12:24:49.138051 17186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:24:49.138855 17186 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:24:49.139073 17186 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Loaded TSK: 0
I20260925 12:24:49.152710 17186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Generated new TSK 1
I20260925 12:24:49.152842 17186 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 12:24:49.229204 17174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:49.229487 17174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:49.229532 17174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:49.230902 17174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:49.230978 17174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:49.231031 17174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:24:49.232820 17174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:43891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=34999
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:49.233132 17174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:49.233398 17174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:49.239504 17201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.239504 17203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.239547 17200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:49.239840 17174 server_base.cc:1034] running on GCE node
I20260925 12:24:49.240084 17174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:49.240413 17174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:49.241600 17174 hybrid_clock.cc:648] HybridClock initialized: now 1790339089241575 us; error 41 us; skew 500 ppm
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:49.244644 17174 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:24:49.245131 17174 webserver.cc:460] Webserver started at http://127.1.47.129:34999/ using document root <none> and password file <none>
I20260925 12:24:49.245386 17174 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:49.245487 17174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:49.247001 17174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 12:24:49.247776 17210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.248013 17174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.248131 17174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "a28eb6075671fee618b85485a6a39ba4"
server_key_iv: "f0ecbb08b26905297ca09c01f3031052"
server_key_version: "encryptionkey@0"
I20260925 12:24:49.248534 17174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:49.259716 17174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:49.262784 17174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:49.262951 17174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:49.263201 17174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:49.263803 17217 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 12:24:49.265367 17174 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 12:24:49.265429 17174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 12:24:49.265463 17174 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 12:24:49.266304 17174 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 12:24:49.266388 17174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.266410 17217 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap starting.
I20260925 12:24:49.273794 17174 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43891
I20260925 12:24:49.274339 17174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:24:49.276734 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17174
I20260925 12:24:49.277010 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:40225
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=44019
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:24:49.277132 17217 log.cc:824] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:49.279758 17217 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:24:49.280076 17217 tablet_bootstrap.cc:492] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap complete.
I20260925 12:24:49.281037 17217 ts_tablet_manager.cc:1397] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260925 12:24:49.282305 17217 raft_consensus.cc:348] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:49.282677 17217 raft_consensus.cc:729] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab337d38bf8d43cc85aa2a703d07e67e, State: Initialized, Role: FOLLOWER
I20260925 12:24:49.282850 17217 consensus_queue.cc:260] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } }
I20260925 12:24:49.283133 17217 ts_tablet_manager.cc:1428] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 12:24:49.283355 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43891 every 8 connection(s)
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:49.289094 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.275337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:57949)
0925 12:24:49.275562 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.275633 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.276729 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.277543 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.277547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.277902 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:49.278180 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.278188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.279912 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.279913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.281144 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.281146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.281938 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.281940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.282082 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.282712 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:49.282722 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:49.284654 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:49.286977 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.286986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.286989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.287374 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.287669 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.287670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.287671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.287827 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.288300 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:49.288307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:49.288495 (+ 188us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.288618 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.288832 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260925 12:24:49.289116 17328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.275725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:57949 (local address 127.1.47.190:38531)
0925 12:24:49.275963 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.275974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.276044 (+ 70us) server_negotiation.cc:407] Connection header received
0925 12:24:49.277057 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.277064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.277127 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.277404 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:49.278318 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.279693 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.281223 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.281820 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.281947 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.284770 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:49.284774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:49.284774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:49.284786 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:49.286879 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.287475 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.287478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.287479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.287594 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.287925 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.287926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.287927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.288224 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:49.288315 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.288724 (+ 409us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.288871 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260925 12:24:49.290328 17325 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:49.290446 17325 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:49.290648 17325 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:49.291435 17136 ts_manager.cc:194] Registered new tserver with Master: ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891)
I20260925 12:24:49.292667 17136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:57949
I20260925 12:24:49.293773 17325 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:49.300103 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.292067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.0.0.1:49422)
0925 12:24:49.295981 (+ 3914us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.296024 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.296094 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.296420 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.296424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.296426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:49.296497 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.296499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.297807 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.297808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.298837 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.298840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.299743 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.299744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.299819 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.299830 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.299986 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.300037 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3878,"thread_start_us":93,"threads_started":1}
I20260925 12:24:49.300144 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.292335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.1.47.129:43891)
0925 12:24:49.296228 (+ 3893us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.296233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.296241 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:49.296274 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.296276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.296291 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.296465 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:49.296580 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.297717 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.298916 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.299640 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.299720 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.299783 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.299999 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.300049 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3862,"thread_start_us":76,"threads_started":1}
I20260925 12:24:49.300591 17259 tablet_service.cc:1514] Processing DeleteTablet for tablet e51213e4d9d142a5886d8ec68067f7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:28 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49422
I20260925 12:24:49.301179 17334 tablet_replica.cc:330] stopping tablet replica
I20260925 12:24:49.301306 17334 raft_consensus.cc:2229] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:49.301412 17334 raft_consensus.cc:2258] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:49.301738 17334 ts_tablet_manager.cc:1905] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:49.304193 17334 ts_tablet_manager.cc:1918] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:49.304260 17334 log.cc:1196] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal/wals/e51213e4d9d142a5886d8ec68067f7df
I20260925 12:24:49.304528 17334 ts_tablet_manager.cc:1939] T e51213e4d9d142a5886d8ec68067f7df P ab337d38bf8d43cc85aa2a703d07e67e: Deleting consensus metadata
I20260925 12:24:49.305045 17122 catalog_manager.cc:4841] TS ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891): tablet e51213e4d9d142a5886d8ec68067f7df (table test-table [id=db159cdbb65f4fe792025a2d4511bd76]) successfully deleted
W20260925 12:24:49.369114 17330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:49.369344 17330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:49.369382 17330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:49.370524 17330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:49.370586 17330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:49.370630 17330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:24:49.372112 17330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:40225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=44019
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:49.372388 17330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:49.372610 17330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:49.377872 17339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.377874 17340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.377868 17342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:49.378127 17330 server_base.cc:1034] running on GCE node
I20260925 12:24:49.378353 17330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:49.378600 17330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:49.379777 17330 hybrid_clock.cc:648] HybridClock initialized: now 1790339089379768 us; error 40 us; skew 500 ppm
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:49.382452 17330 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:24:49.382901 17330 webserver.cc:460] Webserver started at http://127.1.47.130:44019/ using document root <none> and password file <none>
I20260925 12:24:49.383126 17330 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:49.383206 17330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:49.384648 17330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 12:24:49.385232 17349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.385486 17330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.385604 17330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "9b9f27d7afea97306047b1ad9bac7818"
server_key_iv: "3f7af2db7125d9d54392378f53741a7d"
server_key_version: "encryptionkey@0"
I20260925 12:24:49.386000 17330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:49.407814 17330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:49.410185 17330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:49.410408 17330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:49.410678 17330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:49.411005 17330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:49.411060 17330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.411105 17330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:49.411149 17330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.417256 17330 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:40225
I20260925 12:24:49.417382 17462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:40225 every 8 connection(s)
I20260925 12:24:49.417714 17330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:24:49.418372 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17330
I20260925 12:24:49.418694 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:38369
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=39493
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:49.429374 17328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.418572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:53963 (local address 127.1.47.190:38531)
0925 12:24:49.418635 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.418637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.418960 (+ 323us) server_negotiation.cc:407] Connection header received
0925 12:24:49.419559 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.419561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.419579 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.419660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:49.420738 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.421304 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.423096 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.423391 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.423464 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.426414 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:49.426416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:49.426416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:49.426428 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:49.427958 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.428379 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.428380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.428381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.428440 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.428702 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.428703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.428704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.428826 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:49.428900 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.429176 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.429282 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 12:24:49.429584 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.418580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:53963)
0925 12:24:49.418776 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.418838 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.419444 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.419720 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.419724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.420406 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:49.420623 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.420629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.421413 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.421415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.422968 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.422972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.423501 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.423502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.423684 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.424273 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:49.424286 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:49.426296 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:49.428057 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.428064 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.428067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.428301 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.428520 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.428521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.428521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.428637 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.428892 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:49.428899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:49.429049 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.429158 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.429325 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260925 12:24:49.430091 17463 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:49.430186 17463 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:49.430389 17463 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:49.430796 17136 ts_manager.cc:194] Registered new tserver with Master: a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225)
I20260925 12:24:49.431177 17136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:53963
W20260925 12:24:49.510650 17466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:24:49.510916 17466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:24:49.510957 17466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:24:49.512267 17466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:24:49.512331 17466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:24:49.512380 17466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:24:49.513976 17466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:38369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=39493
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:24:49.514253 17466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:24:49.514498 17466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:24:49.520051 17474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.520068 17472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:24:49.520051 17471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:24:49.520076 17466 server_base.cc:1034] running on GCE node
I20260925 12:24:49.520560 17466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:24:49.520815 17466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:24:49.521986 17466 hybrid_clock.cc:648] HybridClock initialized: now 1790339089521978 us; error 36 us; skew 500 ppm
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:24:49.524751 17466 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:24:49.525230 17466 webserver.cc:460] Webserver started at http://127.1.47.131:39493/ using document root <none> and password file <none>
I20260925 12:24:49.525453 17466 fs_manager.cc:359] Metadata directory not provided
I20260925 12:24:49.525530 17466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:24:49.526906 17466 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.527686 17481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.527959 17466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:24:49.528061 17466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "81c8f02e529b49999f2b60824466fcf4"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "0fe73f6156eb7d4344c10373b41fc562"
server_key_iv: "7334633707babfdc732637c4e48fad69"
server_key_version: "encryptionkey@0"
I20260925 12:24:49.528440 17466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:24:49.546883 17466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:24:49.549786 17466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:24:49.549952 17466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:24:49.550236 17466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:24:49.550673 17466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:24:49.550719 17466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.550772 17466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:24:49.550796 17466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.557063 17466 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38369
I20260925 12:24:49.557106 17594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38369 every 8 connection(s)
I20260925 12:24:49.557466 17466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:24:49 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:24:49.565253 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17466
I20260925 12:24:49.567374 17328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.558155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:38253 (local address 127.1.47.190:38531)
0925 12:24:49.558220 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.558222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.558446 (+ 224us) server_negotiation.cc:407] Connection header received
0925 12:24:49.558972 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.558973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.558985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.559055 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:49.560088 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.560585 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.561596 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.561806 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.561888 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.564338 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:49.564340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:49.564340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:49.564348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:49.565950 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.566338 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.566340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.566340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.566398 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.566658 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.566659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.566659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.566779 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:49.566853 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.567136 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.567285 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:24:49.567646 17597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.558143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:38253)
0925 12:24:49.558310 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.558348 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.558866 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.559129 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.559133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.559711 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:24:49.559986 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.559991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.560682 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.560683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.561497 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.561499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.561897 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.561898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.562024 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.562559 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:24:49.562567 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:24:49.564178 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:24:49.566030 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.566037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.566040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.566263 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.566468 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:24:49.566469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:24:49.566469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:24:49.566575 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:24:49.566845 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:24:49.566851 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:24:49.566993 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.567102 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.567317 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260925 12:24:49.568156 17595 heartbeater.cc:347] Connected to a master server at 127.1.47.190:38531
I20260925 12:24:49.568264 17595 heartbeater.cc:464] Registering TS with master...
I20260925 12:24:49.568459 17595 heartbeater.cc:511] Master 127.1.47.190:38531 requested a full tablet report, sending...
I20260925 12:24:49.568861 17136 ts_manager.cc:194] Registered new tserver with Master: 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
I20260925 12:24:49.569314 17136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:38253
I20260925 12:24:49.572705 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.565692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.1.47.190:38531)
0925 12:24:49.565825 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.565828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.565940 (+ 112us) server_negotiation.cc:407] Connection header received
0925 12:24:49.566025 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.566026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.566038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.566114 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:49.566741 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.567330 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.568589 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.568788 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.568864 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.570167 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:49.570168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:49.570169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:49.570177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:49.571511 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.571897 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.571898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.571899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.571971 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.572173 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.572174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.572175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.572291 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:49.572473 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.572511 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.572595 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260925 12:24:49.573537 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:24:49.576417 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.573922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.1.47.190:38531)
0925 12:24:49.573966 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.573968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.574078 (+ 110us) server_negotiation.cc:407] Connection header received
0925 12:24:49.574137 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.574138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.574181 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.574245 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:49.574431 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.574888 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.575679 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.575959 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.576033 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.576200 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:49.576245 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 12:24:49.576370 (+ 125us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41446: token expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 12:24:49.576483 17598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41446: token expired
W20260925 12:24:49.576723 17062 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.47.190:38531)
I20260925 12:24:49.582823 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.577059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.1.47.190:38531)
0925 12:24:49.577097 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.577099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.577107 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:24:49.577126 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.577127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.577135 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.577194 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:24:49.577674 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.578109 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.578817 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.579001 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.579060 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.580303 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:24:49.580304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:24:49.580304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:24:49.580311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:24:49.581629 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.581983 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.581984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.581985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.582041 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:24:49.582294 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:24:49.582295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:24:49.582296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:24:49.582445 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:24:49.582528 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.582686 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.582758 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:24:49.583990 17136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:24:49.585541 17136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:24:49.587841 17259 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea4590ee19548b39c477ab5325bca81 (DEFAULT_TABLE table=test-table [id=9c95daa1e8574ef485a0c3d379846876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:49.588141 17259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea4590ee19548b39c477ab5325bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:49.589655 17217 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap starting.
I20260925 12:24:49.590006 17217 tablet_bootstrap.cc:654] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:49.590790 17217 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: No bootstrap required, opened a new log
I20260925 12:24:49.590854 17217 ts_tablet_manager.cc:1397] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 12:24:49.590968 17217 raft_consensus.cc:348] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.591050 17217 raft_consensus.cc:374] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:49.591104 17217 raft_consensus.cc:729] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab337d38bf8d43cc85aa2a703d07e67e, State: Initialized, Role: FOLLOWER
I20260925 12:24:49.591150 17217 consensus_queue.cc:260] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.591245 17217 ts_tablet_manager.cc:1428] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 12:24:49.591588 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.587346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40225 (local address 127.0.0.1:40396)
0925 12:24:49.587404 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.587415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.587454 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.587921 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.587922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.587923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:49.587980 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.587982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.589349 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.589350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.590414 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.590417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.591315 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.591316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.591382 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.591386 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.591429 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.591519 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":43}
I20260925 12:24:49.591989 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.587487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.1.47.130:40225)
0925 12:24:49.587723 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.587728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.587739 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:24:49.587779 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.587781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.587797 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.587926 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:49.588056 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.589210 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.590481 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.591212 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.591488 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.591814 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.591842 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.591911 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
I20260925 12:24:49.592748 17397 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea4590ee19548b39c477ab5325bca81 (DEFAULT_TABLE table=test-table [id=9c95daa1e8574ef485a0c3d379846876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:49.593106 17397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea4590ee19548b39c477ab5325bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:49.593438 17601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.587346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38369 (local address 127.0.0.1:50790)
0925 12:24:49.587544 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.587556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.587614 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.590098 (+ 2484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.590099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.590101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:49.590150 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.590151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.591360 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.591361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.592350 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.592352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.593206 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.593207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.593269 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.593273 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.593330 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.593370 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":20,"thread_start_us":119,"threads_started":1}
I20260925 12:24:49.593464 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.587552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.1.47.131:38369)
0925 12:24:49.589824 (+ 2272us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.589829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.589841 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:24:49.589879 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.589881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.589909 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.589990 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:49.590233 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.591283 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.592442 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.593112 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.593202 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.593256 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.593343 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.593381 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2227,"thread_start_us":77,"threads_started":1}
I20260925 12:24:49.594141 17529 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea4590ee19548b39c477ab5325bca81 (DEFAULT_TABLE table=test-table [id=9c95daa1e8574ef485a0c3d379846876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:24:49.594477 17529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea4590ee19548b39c477ab5325bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:24:49.596670 17605 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Bootstrap starting.
I20260925 12:24:49.597314 17605 tablet_bootstrap.cc:654] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:49.597724 17606 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Bootstrap starting.
I20260925 12:24:49.597941 17605 log.cc:824] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:49.598353 17606 tablet_bootstrap.cc:654] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Neither blocks nor log segments found. Creating new log.
I20260925 12:24:49.598697 17605 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: No bootstrap required, opened a new log
I20260925 12:24:49.598814 17605 ts_tablet_manager.cc:1397] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:49.598932 17606 log.cc:824] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:24:49.599601 17606 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: No bootstrap required, opened a new log
I20260925 12:24:49.599651 17606 ts_tablet_manager.cc:1397] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:24:49.600669 17605 raft_consensus.cc:348] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.600862 17605 raft_consensus.cc:374] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:49.600914 17605 raft_consensus.cc:729] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a555ccab69e34ae2a9ef04651ff7db0e, State: Initialized, Role: FOLLOWER
I20260925 12:24:49.601049 17605 consensus_queue.cc:260] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.601172 17606 raft_consensus.cc:348] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.601337 17606 raft_consensus.cc:374] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:24:49.601346 17605 ts_tablet_manager.cc:1428] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 12:24:49.601384 17606 raft_consensus.cc:729] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c8f02e529b49999f2b60824466fcf4, State: Initialized, Role: FOLLOWER
I20260925 12:24:49.601498 17463 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:49.601491 17606 consensus_queue.cc:260] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.601752 17606 ts_tablet_manager.cc:1428] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:24:49.601799 17595 heartbeater.cc:503] Master 127.1.47.190:38531 was elected leader, sending a full tablet report...
I20260925 12:24:49.661039 17609 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:24:49.661160 17609 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.661513 17609 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369), ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891)
I20260925 12:24:49.664930 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.661748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38369 (local address 127.1.47.130:44305)
0925 12:24:49.661826 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.661842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.661899 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.662044 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.662045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.662047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:49.662117 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.662118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.662769 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.662770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.663985 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.663987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.664733 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.664733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.664797 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.664800 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.664843 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.664873 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 12:24:49.667635 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.661859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:44305 (local address 127.1.47.131:38369)
0925 12:24:49.661913 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.661915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.661924 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:24:49.661954 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.661955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.661965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.662016 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:49.662191 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.662688 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.664068 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.664651 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.664738 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.664747 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.667524 (+ 2777us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.667571 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 12:24:49.667990 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea4590ee19548b39c477ab5325bca81" candidate_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c8f02e529b49999f2b60824466fcf4" is_pre_election: true
I20260925 12:24:49.668164 17549 raft_consensus.cc:2454] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a555ccab69e34ae2a9ef04651ff7db0e in term 0.
W20260925 12:24:49.668282 17464 tablet.cc:2369] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:24:49.668602 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.661897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.1.47.130:39843)
0925 12:24:49.662040 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.662051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:24:49.662129 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:24:49.662261 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:24:49.662261 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:24:49.662263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:24:49.662333 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.662335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.662983 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.662984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.664098 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:24:49.664100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.668299 (+ 4199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.668309 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:24:49.668400 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.668406 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 12:24:49.668470 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:24:49.668516 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260925 12:24:49.668593 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.661947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:39843 (local address 127.1.47.129:43891)
0925 12:24:49.662005 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.662007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.662116 (+ 109us) server_negotiation.cc:407] Connection header received
0925 12:24:49.662185 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.662186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.662200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.662259 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:24:49.662388 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.662879 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.664170 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.668113 (+ 3943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.668246 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.668256 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.668478 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.668525 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:24:49.668632 17352 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e; no voters:
I20260925 12:24:49.668785 17609 raft_consensus.cc:2781] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:24:49.668962 17609 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:24:49.669029 17609 raft_consensus.cc:3037] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:49.669142 17279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea4590ee19548b39c477ab5325bca81" candidate_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" is_pre_election: true
I20260925 12:24:49.669312 17279 raft_consensus.cc:2454] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a555ccab69e34ae2a9ef04651ff7db0e in term 0.
I20260925 12:24:49.669920 17609 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.670029 17609 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [CANDIDATE]: Term 1 election: Requested vote from peers 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369), ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891)
I20260925 12:24:49.670223 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea4590ee19548b39c477ab5325bca81" candidate_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c8f02e529b49999f2b60824466fcf4"
I20260925 12:24:49.670243 17279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea4590ee19548b39c477ab5325bca81" candidate_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
I20260925 12:24:49.670320 17549 raft_consensus.cc:3037] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:49.670359 17279 raft_consensus.cc:3037] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:24:49.671324 17549 raft_consensus.cc:2454] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a555ccab69e34ae2a9ef04651ff7db0e in term 1.
I20260925 12:24:49.671518 17352 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e; no voters:
I20260925 12:24:49.671648 17279 raft_consensus.cc:2454] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a555ccab69e34ae2a9ef04651ff7db0e in term 1.
I20260925 12:24:49.671747 17609 raft_consensus.cc:2781] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:24:49.671939 17609 raft_consensus.cc:686] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 1 LEADER]: Becoming Leader. State: Replica: a555ccab69e34ae2a9ef04651ff7db0e, State: Running, Role: LEADER
I20260925 12:24:49.672048 17609 consensus_queue.cc:237] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:24:49.672904 17136 catalog_manager.cc:5495] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e reported cstate change: term changed from 0 to 1, leader changed from <none> to a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130). New cstate: current_term: 1 leader_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } health_report { overall_health: UNKNOWN } } }
I20260925 12:24:49.683368 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.681187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.1.47.130:40225)
0925 12:24:49.681228 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:24:49.681230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:24:49.681236 (+ 6us) server_negotiation.cc:407] Connection header received
0925 12:24:49.681263 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:24:49.681264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:24:49.681281 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:24:49.681359 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:24:49.681474 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.681923 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.682611 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:24:49.682799 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:24:49.682856 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:24:49.683029 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:24:49.683123 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:24:49.683179 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:24:49.683223 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 12:24:49.683238 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:24:49.685156 17279 raft_consensus.cc:1261] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Refusing update from remote peer a555ccab69e34ae2a9ef04651ff7db0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:49.685196 17549 raft_consensus.cc:1261] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Refusing update from remote peer a555ccab69e34ae2a9ef04651ff7db0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:24:49.685552 17609 consensus_queue.cc:1035] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:49.685729 17612 consensus_queue.cc:1035] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:24:49.690506 17136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:49.690596 17136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:24:49.691366 17136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Sending DeleteTablet for 3 replicas of tablet 9ea4590ee19548b39c477ab5325bca81
I20260925 12:24:49.691668 17397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea4590ee19548b39c477ab5325bca81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:49 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40396
I20260925 12:24:49.691694 17529 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea4590ee19548b39c477ab5325bca81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:49 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:50790
I20260925 12:24:49.691723 17259 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea4590ee19548b39c477ab5325bca81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:24:49 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:49422
I20260925 12:24:49.691824 17334 tablet_replica.cc:330] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: stopping tablet replica
I20260925 12:24:49.691912 17334 raft_consensus.cc:2229] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:49.691957 17624 tablet_replica.cc:330] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: stopping tablet replica
I20260925 12:24:49.691957 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17174
I20260925 12:24:49.691973 17334 raft_consensus.cc:2258] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:49.692047 17624 raft_consensus.cc:2229] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:24:49.692133 17624 raft_consensus.cc:2258] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:49.692759 17625 tablet_replica.cc:330] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: stopping tablet replica
I20260925 12:24:49.692922 17625 raft_consensus.cc:2229] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:24:49.693076 17625 raft_consensus.cc:2258] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:24:49.693601 17625 ts_tablet_manager.cc:1905] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:49.694240 17624 ts_tablet_manager.cc:1905] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:24:49.696050 17625 ts_tablet_manager.cc:1918] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:49.696201 17625 log.cc:1196] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/9ea4590ee19548b39c477ab5325bca81
I20260925 12:24:49.696300 17624 ts_tablet_manager.cc:1918] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:24:49.696374 17624 log.cc:1196] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/9ea4590ee19548b39c477ab5325bca81
I20260925 12:24:49.696656 17624 ts_tablet_manager.cc:1939] T 9ea4590ee19548b39c477ab5325bca81 P a555ccab69e34ae2a9ef04651ff7db0e: Deleting consensus metadata
I20260925 12:24:49.696748 17625 ts_tablet_manager.cc:1939] T 9ea4590ee19548b39c477ab5325bca81 P 81c8f02e529b49999f2b60824466fcf4: Deleting consensus metadata
I20260925 12:24:49.697369 17122 catalog_manager.cc:4841] TS 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): tablet 9ea4590ee19548b39c477ab5325bca81 (table test-table [id=9c95daa1e8574ef485a0c3d379846876]) successfully deleted
I20260925 12:24:49.697764 17120 catalog_manager.cc:4841] TS a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): tablet 9ea4590ee19548b39c477ab5325bca81 (table test-table [id=9c95daa1e8574ef485a0c3d379846876]) successfully deleted
W20260925 12:24:49.698047 17122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43891 (error 108)
I20260925 12:24:49.698403 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17330
I20260925 12:24:49.698508 17601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:24:49.698366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43891 (local address 127.0.0.1:49424)
0925 12:24:49.698426 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:24:49.698473 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 12:24:49.699158 17122 catalog_manager.cc:4584] TS ab337d38bf8d43cc85aa2a703d07e67e (127.1.47.129:43891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ea4590ee19548b39c477ab5325bca81: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43891: connect: Connection refused (error 111)
I20260925 12:24:49.703682 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17466
I20260925 12:24:49.709594 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17103
I20260925 12:25:09.714994 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--webserver_interface=127.1.47.190
--webserver_port=33699
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:09.797770 17627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:09.797997 17627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:09.798022 17627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:09.799257 17627 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 12:25:09.799304 17627 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 12:25:09.799316 17627 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 12:25:09.799328 17627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:09.799342 17627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:09.799404 17627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:09.799422 17627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:09.799443 17627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:09.800972 17627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:09.801255 17627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:09.801529 17627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:09.806915 17635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:09.806965 17627 server_base.cc:1034] running on GCE node
W20260925 12:25:09.806931 17632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:09.806931 17633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:09.807480 17627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:09.807765 17627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:09.809017 17627 hybrid_clock.cc:648] HybridClock initialized: now 1790339109809004 us; error 50 us; skew 500 ppm
Sep 25 12:25:09 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:09.811753 17627 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:09.812301 17627 webserver.cc:460] Webserver started at http://127.1.47.190:33699/ using document root <none> and password file <none>
I20260925 12:25:09.812569 17627 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:09.812654 17627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:09.814210 17627 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:25:09.815022 17642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:09.815235 17627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:09.815354 17627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "48c9965cf75e48798325e71fa8270b04"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "4bd38bc98f09cb7cf39c57edd25f739d"
server_key_iv: "14ea292611c7a054a11a34755a882f64"
server_key_version: "encryptionkey@0"
I20260925 12:25:09.815775 17627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:09.842971 17627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:09.845502 17627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:09.845712 17627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:09.849912 17627 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:38531
I20260925 12:25:09.849952 17694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:38531 every 8 connection(s)
I20260925 12:25:09.850373 17627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:09.852190 17695 sys_catalog.cc:263] Verifying existing consensus state
I20260925 12:25:09.852892 17695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap starting.
I20260925 12:25:09.860757 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17627
I20260925 12:25:09.861147 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:43891
--local_ip_for_outbound_sockets=127.1.47.129
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=34999
--webserver_interface=127.1.47.129
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:25:09.862604 17695 log.cc:824] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:09.865674 17695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:25:09.866051 17695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Bootstrap complete.
I20260925 12:25:09.868675 17695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:25:09.868938 17695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Initialized, Role: FOLLOWER
I20260925 12:25:09.869140 17695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:25:09.869222 17695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:09.869282 17695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:09.869344 17695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 2 FOLLOWER]: Advancing to term 3
I20260925 12:25:09.870501 17695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:25:09.870605 17695 leader_election.cc:302] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c9965cf75e48798325e71fa8270b04; no voters:
I20260925 12:25:09.870874 17695 leader_election.cc:288] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 12:25:09.870916 17699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 12:25:09.871050 17699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [term 3 LEADER]: Becoming Leader. State: Replica: 48c9965cf75e48798325e71fa8270b04, State: Running, Role: LEADER
I20260925 12:25:09.871150 17699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } }
I20260925 12:25:09.871182 17695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:09.871466 17699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c9965cf75e48798325e71fa8270b04. Latest consensus state: current_term: 3 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:25:09.871565 17699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:09.871690 17700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "48c9965cf75e48798325e71fa8270b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c9965cf75e48798325e71fa8270b04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 38531 } } }
I20260925 12:25:09.871778 17700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:09.871923 17703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:09.872923 17703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:09.873060 17703 catalog_manager.cc:1259] Loaded cluster ID: 638f226b37ed4b80a4abcd0b9e849fa5
I20260925 12:25:09.873106 17703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:09.874418 17703 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:09.874634 17703 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Loaded TSK: 0
I20260925 12:25:09.874706 17703 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 48c9965cf75e48798325e71fa8270b04: Loaded TSK: 1
F20260925 12:25:09.875008 17703 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f92270829ad google::LogMessage::Fail() at ??:0
@ 0x7f922708588a google::LogMessage::SendToLog() at ??:0
@ 0x7f92270824de google::LogMessage::Flush() at ??:0
@ 0x7f9227083a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f92297ef104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f92297ecd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f922efdc1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f922efdc48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f922efdc594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f922efa2c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f922efacad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9228f500d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9228f45954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9229da96db start_thread at ??:0
@ 0x7f9225b8061f clone at ??:0
W20260925 12:25:09.958372 17697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:09.958626 17697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:09.958668 17697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:09.959913 17697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:09.959988 17697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:09.960034 17697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:09.961580 17697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:43891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=34999
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:09.961864 17697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:09.962106 17697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:09.967134 17736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:09.967159 17739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:09.967126 17737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:09.967460 17697 server_base.cc:1034] running on GCE node
I20260925 12:25:09.967667 17697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:09.967964 17697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:09.969153 17697 hybrid_clock.cc:648] HybridClock initialized: now 1790339109969139 us; error 28 us; skew 500 ppm
Sep 25 12:25:09 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:09.972222 17697 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:25:09.972725 17697 webserver.cc:460] Webserver started at http://127.1.47.129:34999/ using document root <none> and password file <none>
I20260925 12:25:09.972963 17697 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:09.973031 17697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:09.974489 17697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:09.975243 17746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:09.975442 17697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:09.975535 17697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "ab337d38bf8d43cc85aa2a703d07e67e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "a28eb6075671fee618b85485a6a39ba4"
server_key_iv: "f0ecbb08b26905297ca09c01f3031052"
server_key_version: "encryptionkey@0"
I20260925 12:25:09.975919 17697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:09.987020 17697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:09.989432 17697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:09.989598 17697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:09.989825 17697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:09.990285 17753 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 12:25:09.992401 17697 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 12:25:09.992483 17697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 12:25:09.992532 17697 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 12:25:09.993161 17697 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 12:25:09.993214 17697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 12:25:09.993304 17753 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap starting.
I20260925 12:25:10.000442 17697 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43891
I20260925 12:25:10.000604 17860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43891 every 8 connection(s)
I20260925 12:25:10.000891 17697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:10.005075 17753 log.cc:824] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:10.006892 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17697
I20260925 12:25:10.007150 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:40225
--local_ip_for_outbound_sockets=127.1.47.130
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=44019
--webserver_interface=127.1.47.130
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:25:10.007939 17753 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 12:25:10.008248 17753 tablet_bootstrap.cc:492] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Bootstrap complete.
I20260925 12:25:10.009508 17753 ts_tablet_manager.cc:1397] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260925 12:25:10.010996 17753 raft_consensus.cc:348] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:10.011360 17753 raft_consensus.cc:729] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab337d38bf8d43cc85aa2a703d07e67e, State: Initialized, Role: FOLLOWER
I20260925 12:25:10.011543 17753 consensus_queue.cc:260] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:10.011822 17753 ts_tablet_manager.cc:1428] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260925 12:25:10.090111 17863 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:10.001684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:42515)
0925 12:25:10.001913 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.001953 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:10.002408 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:10.089893 (+ 87485us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
W20260925 12:25:10.090369 17861 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:38531 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:38531: Client connection negotiation failed: client connection to 127.1.47.190:38531: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 12:25:10.090734 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.090604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:53915)
0925 12:25:10.090663 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.090707 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 12:25:10.090791 17865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:10.091051 17865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260925 12:25:10.091068 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.090980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:33725)
0925 12:25:10.091023 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.091046 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 12:25:10.091099 17865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:10.091184 17861 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:25:10.092474 17865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:10.092540 17865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:10.092586 17865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:10.094069 17865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:40225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=44019
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:10.094347 17865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:10.094587 17865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:10.099756 17874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:10.099747 17871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:10.099839 17872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:10.099951 17865 server_base.cc:1034] running on GCE node
I20260925 12:25:10.100286 17865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:10.100562 17865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:10.101729 17865 hybrid_clock.cc:648] HybridClock initialized: now 1790339110101693 us; error 57 us; skew 500 ppm
Sep 25 12:25:10 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:10.104693 17865 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:25:10.105206 17865 webserver.cc:460] Webserver started at http://127.1.47.130:44019/ using document root <none> and password file <none>
I20260925 12:25:10.105461 17865 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:10.105533 17865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:10.106951 17865 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:10.107693 17881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.108109 17865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:10.108248 17865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "a555ccab69e34ae2a9ef04651ff7db0e"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "9b9f27d7afea97306047b1ad9bac7818"
server_key_iv: "3f7af2db7125d9d54392378f53741a7d"
server_key_version: "encryptionkey@0"
I20260925 12:25:10.108624 17865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:10.131649 17865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:10.134135 17865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:10.134253 17865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:10.134459 17865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:10.134789 17865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:10.134826 17865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.134850 17865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:10.134862 17865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.140618 17865 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:40225
I20260925 12:25:10.140709 17994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:40225 every 8 connection(s)
I20260925 12:25:10.141048 17865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:25:10.142158 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.141778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:50491)
0925 12:25:10.141932 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.142005 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260925 12:25:10.142553 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17865
W20260925 12:25:10.142714 17995 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:38531 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:38531: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
I20260925 12:25:10.142843 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:38369
--local_ip_for_outbound_sockets=127.1.47.131
--tserver_master_addrs=127.1.47.190:38531
--webserver_port=39493
--webserver_interface=127.1.47.131
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:25:10.143054 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.142945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:58965)
0925 12:25:10.143005 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.143030 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 12:25:10.143373 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.143289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:33087)
0925 12:25:10.143330 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.143354 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 12:25:10.143482 17995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 12:25:10.224588 17998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:10.224808 17998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:10.224840 17998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:10.226025 17998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:10.226069 17998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:10.226096 17998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:10.227607 17998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:38369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=39493
--tserver_master_addrs=127.1.47.190:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:10.227885 17998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:10.228133 17998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:10.233904 18003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:10.233899 18006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:10.233937 18004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:10.234122 17998 server_base.cc:1034] running on GCE node
I20260925 12:25:10.234390 17998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:10.234649 17998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:10.235819 17998 hybrid_clock.cc:648] HybridClock initialized: now 1790339110235799 us; error 42 us; skew 500 ppm
Sep 25 12:25:10 dist-test-slave-l9ww krb5kdc[16555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:10.239272 17998 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:25:10.239729 17998 webserver.cc:460] Webserver started at http://127.1.47.131:39493/ using document root <none> and password file <none>
I20260925 12:25:10.240031 17998 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:10.240101 17998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:10.241513 17998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:10.242265 18013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.242482 17998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:10.242584 17998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "81c8f02e529b49999f2b60824466fcf4"
format_stamp: "Formatted at 2026-09-25 12:24:28 on dist-test-slave-l9ww"
server_key: "0fe73f6156eb7d4344c10373b41fc562"
server_key_iv: "7334633707babfdc732637c4e48fad69"
server_key_version: "encryptionkey@0"
I20260925 12:25:10.242966 17998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:10.253157 17998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:10.255431 17998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:10.255590 17998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:10.255823 17998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:10.256160 17998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:10.256217 17998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.256268 17998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:10.256299 17998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:10.262408 17998 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38369
I20260925 12:25:10.262456 18126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38369 every 8 connection(s)
I20260925 12:25:10.262799 17998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:10.263906 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.263513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:48909)
0925 12:25:10.263664 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.263742 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
W20260925 12:25:10.264339 18127 heartbeater.cc:644] Failed to heartbeat to 127.1.47.190:38531 (0 consecutive failures): Network error: Failed to ping master at 127.1.47.190:38531: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
I20260925 12:25:10.264649 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.264559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:46337)
0925 12:25:10.264595 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.264632 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 12:25:10.264989 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:10.264910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:35815)
0925 12:25:10.264954 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:10.264975 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:25:10.265141 18127 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 12:25:10.268684 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 17998
I20260925 12:25:11.091913 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.091580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:59933)
0925 12:25:11.091746 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:11.091791 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260925 12:25:11.144073 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.143799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:44539)
0925 12:25:11.143978 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:11.144017 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260925 12:25:11.265830 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.265563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:50037)
0925 12:25:11.265720 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:11.265762 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260925 12:25:11.538856 18134 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:11.538990 18134 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:11.539397 18134 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
Sep 25 12:25:11 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.131@KRBTEST.COM
Sep 25 12:25:11 dist-test-slave-l9ww krb5kdc[16555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.130@KRBTEST.COM
I20260925 12:25:11.550496 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.539666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38369 (local address 127.1.47.129:52469)
0925 12:25:11.539800 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:11.539808 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:11.539849 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:11.540731 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:11.540732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:11.541103 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:11.541305 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:11.541310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.542407 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.542408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:11.543205 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:11.543207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.543843 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:11.543945 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:11.544553 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:11.544564 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:11.546423 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:11.548686 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:11.548692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:11.548695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:11.549002 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:11.549422 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549422 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:11.549423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:11.549521 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:11.550023 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:11.550029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:11.550158 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 12:25:11.550264 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 12:25:11.550394 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":78,"thread_start_us":81,"threads_started":1}
I20260925 12:25:11.550691 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.539805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:52469 (local address 127.1.47.131:38369)
0925 12:25:11.540001 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:11.540009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:11.540037 (+ 28us) server_negotiation.cc:407] Connection header received
0925 12:25:11.540472 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:11.540474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:11.540516 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:11.540676 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:11.541441 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.542299 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543295 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.543734 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543889 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:11.546530 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:11.546535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:11.546536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:11.546551 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:11.548596 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549103 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:11.549106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:11.549106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:11.549335 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549626 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:11.549627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:11.549627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:11.549954 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:11.550014 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:11.550447 (+ 433us) server_negotiation.cc:299] Negotiation successful
0925 12:25:11.550578 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260925 12:25:11.550799 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.539822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:49647 (local address 127.1.47.130:40225)
0925 12:25:11.539991 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:11.540001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:11.540028 (+ 27us) server_negotiation.cc:407] Connection header received
0925 12:25:11.540487 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:11.540489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:11.540524 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:11.540673 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:11.541444 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.542296 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543335 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.543712 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543791 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:11.547170 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:11.547174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:11.547175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:11.547186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:11.549132 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549588 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:11.549590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:11.549591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:11.549677 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549867 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:11.549868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:11.549868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:11.550132 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:11.550215 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:11.550561 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 12:25:11.550701 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260925 12:25:11.550674 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:11.539584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:40225 (local address 127.1.47.129:49647)
0925 12:25:11.539668 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:11.539683 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:11.539749 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:11.540709 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:11.540712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:11.541102 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:11.541305 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:11.541310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.542424 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.542425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:11.543213 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:11.543215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:11.543799 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:11.543801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:11.543936 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:11.544582 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:11.544586 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:11.547027 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:11.549303 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:11.549305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:11.549511 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:11.549744 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:11.549745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:11.549745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:11.549806 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:11.550231 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:11.550231 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:11.550401 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 12:25:11.550479 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 12:25:11.550576 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":2}
W20260925 12:25:11.551779 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:11.551889 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:11.551934 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:11.552028 18134 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:12.092816 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:12.092510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:57759)
0925 12:25:12.092713 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:12.092760 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":65,"threads_started":1}
I20260925 12:25:12.144930 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:12.144667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:42339)
0925 12:25:12.144851 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:12.144888 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":125,"threads_started":1}
I20260925 12:25:12.266664 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:12.266416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:40341)
0925 12:25:12.266581 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:12.266624 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260925 12:25:13.093695 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:13.093434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:43177)
0925 12:25:13.093584 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:13.093630 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260925 12:25:13.145794 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:13.145562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:41793)
0925 12:25:13.145702 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:13.145748 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260925 12:25:13.267489 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:13.267215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:42865)
0925 12:25:13.267385 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:13.267436 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260925 12:25:13.365208 18144 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:13.365312 18144 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:13.365499 18144 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
W20260925 12:25:13.366001 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:13.366135 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:13.366271 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:13.366381 18144 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:14.094481 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:14.094187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:55727)
0925 12:25:14.094397 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:14.094434 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
I20260925 12:25:14.146711 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:14.146486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:60295)
0925 12:25:14.146632 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:14.146665 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260925 12:25:14.268389 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:14.268141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:58021)
0925 12:25:14.268292 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:14.268336 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260925 12:25:15.095376 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:15.095093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:37563)
0925 12:25:15.095273 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:15.095315 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":73,"threads_started":1}
I20260925 12:25:15.147409 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:15.147183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:48117)
0925 12:25:15.147319 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:15.147355 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260925 12:25:15.269466 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:15.269180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:53071)
0925 12:25:15.269373 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:15.269418 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260925 12:25:15.317723 18151 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:15.317824 18151 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:15.318005 18151 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
W20260925 12:25:15.318454 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:15.318540 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:15.318590 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:15.318675 18151 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:16.096273 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:16.095996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:45801)
0925 12:25:16.096154 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:16.096195 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260925 12:25:16.148371 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:16.148145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:52503)
0925 12:25:16.148280 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:16.148323 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260925 12:25:16.270354 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:16.270068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:42705)
0925 12:25:16.270245 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:16.270299 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260925 12:25:17.097218 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:17.096941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:49503)
0925 12:25:17.097120 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:17.097168 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260925 12:25:17.149099 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:17.148885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:39235)
0925 12:25:17.149015 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:17.149055 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260925 12:25:17.271133 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:17.270902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:54713)
0925 12:25:17.271029 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:17.271067 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260925 12:25:18.098135 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:18.097849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:57561)
0925 12:25:18.098045 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:18.098089 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
I20260925 12:25:18.149883 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:18.149607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:36017)
0925 12:25:18.149783 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:18.149831 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260925 12:25:18.271991 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:18.271676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:34589)
0925 12:25:18.271834 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:18.271937 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260925 12:25:18.477572 18161 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:18.477676 18161 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:18.477860 18161 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
W20260925 12:25:18.478266 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:18.478381 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:18.478459 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:18.478561 18161 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:19.099015 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:19.098759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:54947)
0925 12:25:19.098919 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:19.098967 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260925 12:25:19.150743 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:19.150480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:58811)
0925 12:25:19.150638 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:19.150695 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
I20260925 12:25:19.272710 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:19.272444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:56387)
0925 12:25:19.272618 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:19.272657 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260925 12:25:20.099761 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:20.099533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:37391)
0925 12:25:20.099666 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:20.099704 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260925 12:25:20.151551 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:20.151288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:36771)
0925 12:25:20.151443 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:20.151502 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260925 12:25:20.273528 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:20.273278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:56235)
0925 12:25:20.273433 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:20.273478 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260925 12:25:21.100566 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:21.100343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:40207)
0925 12:25:21.100483 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:21.100519 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260925 12:25:21.152431 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:21.152174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:34973)
0925 12:25:21.152338 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:21.152382 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
I20260925 12:25:21.274397 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:21.274142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:36187)
0925 12:25:21.274286 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:21.274332 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260925 12:25:22.101410 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:22.101150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:34741)
0925 12:25:22.101315 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:22.101360 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260925 12:25:22.153256 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:22.152993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:47527)
0925 12:25:22.153130 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:22.153186 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260925 12:25:22.275224 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:22.274985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:43095)
0925 12:25:22.275123 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:22.275173 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260925 12:25:22.931999 18174 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:22.932097 18174 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:22.932276 18174 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
W20260925 12:25:22.932729 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:22.932821 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:22.932921 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:22.933022 18174 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:23.102286 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:23.102011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:53915)
0925 12:25:23.102188 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:23.102237 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260925 12:25:23.154100 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:23.153839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:32883)
0925 12:25:23.153996 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:23.154047 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":62,"threads_started":1}
I20260925 12:25:23.276026 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:23.275737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:52523)
0925 12:25:23.275949 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:23.275979 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260925 12:25:24.103222 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:24.102973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:50177)
0925 12:25:24.103124 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:24.103174 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260925 12:25:24.155110 18179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:24.154822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:53529)
0925 12:25:24.155021 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:24.155064 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260925 12:25:24.276679 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:24.276475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:57865)
0925 12:25:24.276598 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:24.276630 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260925 12:25:24.446287 18181 raft_consensus.cc:482] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:24.446374 18181 raft_consensus.cc:504] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a555ccab69e34ae2a9ef04651ff7db0e" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 40225 } } peers { permanent_uuid: "81c8f02e529b49999f2b60824466fcf4" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38369 } } peers { permanent_uuid: "ab337d38bf8d43cc85aa2a703d07e67e" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43891 } }
I20260925 12:25:24.446627 18181 leader_election.cc:288] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225), 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369)
W20260925 12:25:24.447120 17747 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a555ccab69e34ae2a9ef04651ff7db0e (127.1.47.130:40225): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
W20260925 12:25:24.447213 17749 leader_election.cc:341] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c8f02e529b49999f2b60824466fcf4 (127.1.47.131:38369): Not found: Tablet not found: 9ea4590ee19548b39c477ab5325bca81
I20260925 12:25:24.447252 17747 leader_election.cc:302] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab337d38bf8d43cc85aa2a703d07e67e; no voters: 81c8f02e529b49999f2b60824466fcf4, a555ccab69e34ae2a9ef04651ff7db0e
I20260925 12:25:24.447340 18181 raft_consensus.cc:2726] T 9ea4590ee19548b39c477ab5325bca81 P ab337d38bf8d43cc85aa2a703d07e67e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 12:25:25.104102 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:25.103842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.129:41865)
0925 12:25:25.104023 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:25.104055 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260925 12:25:25.156075 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:25.155765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.130:34041)
0925 12:25:25.155951 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:25.156003 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":138,"threads_started":1}
I20260925 12:25:25.273252 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17697
I20260925 12:25:25.277561 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:25.277302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:38531 (local address 127.1.47.131:58919)
0925 12:25:25.277451 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:25.277500 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.190:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":53,"threads_started":1}
I20260925 12:25:25.278378 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17865
I20260925 12:25:25.282907 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 17998
2026-09-25T12:25:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58247 ms)
[----------] 30 tests from SecurityITest (236315 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:25 dist-test-slave-l9ww krb5kdc[18187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:25 dist-test-slave-l9ww krb5kdc[18187](info): set up 2 sockets
Sep 25 12:25:25 dist-test-slave-l9ww krb5kdc[18187](info): commencing operation
krb5kdc: starting...
W20260925 12:25:26.331586 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:26.454571 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:39955
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:26.536696 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:26.536986 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:26.537029 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:26.538216 18200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:26.538304 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:26.538332 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:26.538362 18200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:26.538390 18200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:26.539932 18200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:39955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:26.540228 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:26.540474 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:26.545310 18205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:26.545367 18208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:26.545320 18206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:26.545518 18200 server_base.cc:1034] running on GCE node
I20260925 12:25:26.545763 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:26.546062 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:26.547209 18200 hybrid_clock.cc:648] HybridClock initialized: now 1790339126547187 us; error 35 us; skew 500 ppm
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:26.549866 18200 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:26.550372 18200 webserver.cc:460] Webserver started at http://127.1.47.190:38391/ using document root <none> and password file <none>
I20260925 12:25:26.550623 18200 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:26.550668 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:26.550801 18200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:26.551940 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "c8de2ab63ff846ed858a504d3628faab"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "a5a7d2a49653ae3ee103f95d4774c2c1"
server_key_iv: "69c85045844b6c2311c60b8ac2376246"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.552318 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "c8de2ab63ff846ed858a504d3628faab"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "a5a7d2a49653ae3ee103f95d4774c2c1"
server_key_iv: "69c85045844b6c2311c60b8ac2376246"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.553701 18200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:26.554507 18215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.554714 18200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:26.554824 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "c8de2ab63ff846ed858a504d3628faab"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "a5a7d2a49653ae3ee103f95d4774c2c1"
server_key_iv: "69c85045844b6c2311c60b8ac2376246"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.554914 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:26.566951 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:26.569250 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:26.569425 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:26.573745 18200 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:39955
I20260925 12:25:26.573787 18267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:39955 every 8 connection(s)
I20260925 12:25:26.574177 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:26.575143 18268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:26.577786 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: Bootstrap starting.
I20260925 12:25:26.578437 18268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:26.578763 18268 log.cc:824] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:26.579615 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: No bootstrap required, opened a new log
I20260925 12:25:26.579675 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18200
I20260925 12:25:26.579779 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:26.579989 1214 external_mini_cluster.cc:1428] Setting key 8f8df88ebc798414cb29d3776d5ee8eb
I20260925 12:25:26.582523 18268 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } }
I20260925 12:25:26.582759 18268 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:26.582813 18268 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8de2ab63ff846ed858a504d3628faab, State: Initialized, Role: FOLLOWER
I20260925 12:25:26.582978 18268 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } }
I20260925 12:25:26.583068 18268 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:26.583105 18268 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:26.583171 18268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:26.584051 18268 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } }
I20260925 12:25:26.584185 18268 leader_election.cc:302] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8de2ab63ff846ed858a504d3628faab; no voters:
I20260925 12:25:26.584390 18268 leader_election.cc:288] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:26.584456 18273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:26.584626 18273 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [term 1 LEADER]: Becoming Leader. State: Replica: c8de2ab63ff846ed858a504d3628faab, State: Running, Role: LEADER
I20260925 12:25:26.584709 18268 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:26.584719 18273 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } }
I20260925 12:25:26.585137 18273 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8de2ab63ff846ed858a504d3628faab. Latest consensus state: current_term: 1 leader_uuid: "c8de2ab63ff846ed858a504d3628faab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } } }
I20260925 12:25:26.585210 18273 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:26.585388 18275 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8de2ab63ff846ed858a504d3628faab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8de2ab63ff846ed858a504d3628faab" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 39955 } } }
I20260925 12:25:26.585484 18275 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:26.585642 18277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:26.586542 18277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:26.588312 18277 catalog_manager.cc:1347] Generated new cluster ID: 0db0a2278410474a8d50972f3529b3c8
I20260925 12:25:26.588366 18277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:26.593536 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:26.580586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.1.47.190:39955)
0925 12:25:26.580793 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:26.580804 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:26.580844 (+ 40us) server_negotiation.cc:407] Connection header received
0925 12:25:26.581518 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:26.581529 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:26.581603 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:26.581785 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:26.582382 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.584316 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.585634 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.586315 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.586414 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:26.589416 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:26.589421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:26.589421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:26.589434 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:26.591692 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.592150 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.592154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.592154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.592282 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.592512 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.592513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.592514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.592871 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:26.593044 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:26.593138 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 12:25:26.593295 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260925 12:25:26.612135 18277 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:26.613039 18277 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:26.627677 18277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: Generated new TSK 0
I20260925 12:25:26.627992 18277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:25:26.659179 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:26.743785 18294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:26.744134 18294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:26.744174 18294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:26.745436 18294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:26.745481 18294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:26.745512 18294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:26.747000 18294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:26.747205 18294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:26.747402 18294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:26.752604 18302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:26.752647 18300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:26.752650 18299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:26.752995 18294 server_base.cc:1034] running on GCE node
I20260925 12:25:26.753211 18294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:26.753489 18294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:26.754664 18294 hybrid_clock.cc:648] HybridClock initialized: now 1790339126754653 us; error 48 us; skew 500 ppm
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:26.757419 18294 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:25:26.757956 18294 webserver.cc:460] Webserver started at http://127.1.47.129:37833/ using document root <none> and password file <none>
I20260925 12:25:26.758210 18294 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:26.758281 18294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:26.758438 18294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:26.759601 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "55c898a9e79a4038b8039abbd51a62fd"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "d2a56e31354b84e7089f051a7ecd87d1"
server_key_iv: "d07189829bc0567b859c0b18d7f2f7ce"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.760053 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "55c898a9e79a4038b8039abbd51a62fd"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "d2a56e31354b84e7089f051a7ecd87d1"
server_key_iv: "d07189829bc0567b859c0b18d7f2f7ce"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.761554 18294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:26.762329 18309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.762503 18294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:26.762603 18294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "55c898a9e79a4038b8039abbd51a62fd"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "d2a56e31354b84e7089f051a7ecd87d1"
server_key_iv: "d07189829bc0567b859c0b18d7f2f7ce"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.762688 18294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:26.780684 18294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:26.783077 18294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:26.783284 18294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:26.783568 18294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:26.783977 18294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:26.784039 18294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.784087 18294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:26.784118 18294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.790978 18294 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:34485
I20260925 12:25:26.791046 18422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:34485 every 8 connection(s)
I20260925 12:25:26.791522 18294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:26.795090 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18294
I20260925 12:25:26.795202 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:26.795428 1214 external_mini_cluster.cc:1428] Setting key f88f441b1f61aecd22b52f3054e7adfb
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:26.803368 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:26.792474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:57627 (local address 127.1.47.190:39955)
0925 12:25:26.792540 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:26.792542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:26.792797 (+ 255us) server_negotiation.cc:407] Connection header received
0925 12:25:26.793346 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:26.793348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:26.793359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:26.793433 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:26.794482 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.795111 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.796639 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.796979 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.797054 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:26.799783 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:26.799785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:26.799785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:26.799794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:26.801722 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.802230 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.802231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.802232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.802303 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.802628 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.802630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.802630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.802799 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:26.802876 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:26.803152 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 12:25:26.803247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:25:26.803462 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:26.792436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:39955 (local address 127.1.47.129:57627)
0925 12:25:26.792638 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:26.792696 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:26.793225 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:26.793523 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:26.793527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:26.794173 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:26.794390 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:26.794395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.795251 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.795252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:26.796548 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:26.796550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.797080 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.797081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:26.797219 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:26.797876 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:26.797890 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:26.799678 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:26.801841 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:26.801852 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:26.801855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:26.802141 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:26.802402 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:26.802403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:26.802403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:26.802550 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:26.802875 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:26.802880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:26.803012 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 12:25:26.803100 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 12:25:26.803241 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":122,"threads_started":1}
I20260925 12:25:26.804095 18423 heartbeater.cc:347] Connected to a master server at 127.1.47.190:39955
I20260925 12:25:26.804215 18423 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:26.804425 18423 heartbeater.cc:511] Master 127.1.47.190:39955 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
I20260925 12:25:26.805025 18232 ts_manager.cc:194] Registered new tserver with Master: 55c898a9e79a4038b8039abbd51a62fd (127.1.47.129:34485)
I20260925 12:25:26.805636 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:57627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:25:26.823751 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:26.906399 18428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:26.906623 18428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:26.906649 18428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:26.907967 18428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:26.908015 18428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:26.908043 18428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:26.909538 18428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:26.909796 18428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:26.910056 18428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:26.915479 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:26.915490 18436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:26.915688 18428 server_base.cc:1034] running on GCE node
W20260925 12:25:26.915813 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:26.916118 18428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:26.916414 18428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:26.917610 18428 hybrid_clock.cc:648] HybridClock initialized: now 1790339126917604 us; error 39 us; skew 500 ppm
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:26.920678 18428 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:25:26.921113 18428 webserver.cc:460] Webserver started at http://127.1.47.130:33189/ using document root <none> and password file <none>
I20260925 12:25:26.921321 18428 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:26.921366 18428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:26.921478 18428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:26.922623 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "22c54555893c41c5a07f055270e3964d"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "8b23d4e5b070401fb044a218408731f6"
server_key_iv: "5803967ca4591cb441f9167590ba3839"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.922955 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "22c54555893c41c5a07f055270e3964d"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "8b23d4e5b070401fb044a218408731f6"
server_key_iv: "5803967ca4591cb441f9167590ba3839"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.924424 18428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:26.925150 18443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.925333 18428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:25:26.925408 18428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "22c54555893c41c5a07f055270e3964d"
format_stamp: "Formatted at 2026-09-25 12:25:26 on dist-test-slave-l9ww"
server_key: "8b23d4e5b070401fb044a218408731f6"
server_key_iv: "5803967ca4591cb441f9167590ba3839"
server_key_version: "encryptionkey@0"
I20260925 12:25:26.925467 18428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:26.933594 18428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:26.936326 18428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:26.936456 18428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:26.936667 18428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:26.936944 18428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:26.936978 18428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.937002 18428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:26.937017 18428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:26.943414 18428 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:46701
I20260925 12:25:26.943496 18556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:46701 every 8 connection(s)
I20260925 12:25:26.943926 18428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:25:26.949045 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18428
I20260925 12:25:26.949208 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:26.949432 1214 external_mini_cluster.cc:1428] Setting key a109fecf9a5a6a359a6e88326aad1bdc
Sep 25 12:25:26 dist-test-slave-l9ww krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:26.955487 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:26.944744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:44795 (local address 127.1.47.190:39955)
0925 12:25:26.944806 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:26.944807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:26.945033 (+ 226us) server_negotiation.cc:407] Connection header received
0925 12:25:26.945608 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:26.945610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:26.945623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:26.945676 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:26.946702 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.947216 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.948375 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.948572 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.948659 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:26.951837 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:26.951838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:26.951839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:26.951846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:26.953716 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.954187 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.954189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.954190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.954258 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:26.954693 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:26.954694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:26.954694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:26.954855 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:26.954958 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:26.955249 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 12:25:26.955344 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 12:25:26.955523 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:26.944732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:39955 (local address 127.1.47.130:44795)
0925 12:25:26.944905 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:26.944943 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:26.945485 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:26.945756 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:26.945760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:26.946364 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:26.946577 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:26.946582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.947336 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.947343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:26.948274 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:26.948277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:26.948671 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:26.948672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:26.948799 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:26.949571 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:26.949581 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:26.951718 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:26.953826 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:26.953834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:26.953836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:26.954096 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:26.954335 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:26.954336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:26.954336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:26.954604 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:26.954956 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:26.954961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:26.955098 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 12:25:26.955192 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 12:25:26.955339 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260925 12:25:26.956435 18557 heartbeater.cc:347] Connected to a master server at 127.1.47.190:39955
I20260925 12:25:26.956542 18557 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:26.956761 18557 heartbeater.cc:511] Master 127.1.47.190:39955 requested a full tablet report, sending...
I20260925 12:25:26.957250 18232 ts_manager.cc:194] Registered new tserver with Master: 22c54555893c41c5a07f055270e3964d (127.1.47.130:46701)
I20260925 12:25:26.957671 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:44795
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:25:26.979233 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:27.061522 18562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:27.061859 18562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:27.061901 18562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:27.063098 18562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:27.063171 18562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:27.063217 18562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:27.064886 18562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:27.065160 18562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:27.065410 18562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:27.070501 18568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.070680 18567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.070612 18570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:27.070957 18562 server_base.cc:1034] running on GCE node
I20260925 12:25:27.071169 18562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:27.071444 18562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:27.072660 18562 hybrid_clock.cc:648] HybridClock initialized: now 1790339127072656 us; error 38 us; skew 500 ppm
Sep 25 12:25:27 dist-test-slave-l9ww krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:27.075400 18562 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:25:27.075826 18562 webserver.cc:460] Webserver started at http://127.1.47.131:46403/ using document root <none> and password file <none>
I20260925 12:25:27.076076 18562 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:27.076122 18562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:27.076215 18562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:27.077263 18562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "f840db609a984b8fa9f96c7ba42cd85e"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "f6b30b98abd9c29aec92e55210c90400"
server_key_iv: "2bb262995c0d8b12be3632923ec161b0"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.077569 18562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "f840db609a984b8fa9f96c7ba42cd85e"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "f6b30b98abd9c29aec92e55210c90400"
server_key_iv: "2bb262995c0d8b12be3632923ec161b0"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.078876 18562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:27.079682 18577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.079913 18562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:27.080008 18562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "f840db609a984b8fa9f96c7ba42cd85e"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "f6b30b98abd9c29aec92e55210c90400"
server_key_iv: "2bb262995c0d8b12be3632923ec161b0"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.080067 18562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:27.103157 18562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:27.106362 18562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:27.106554 18562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:27.106864 18562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:27.107231 18562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:27.107270 18562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.107326 18562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:27.107355 18562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.114010 18562 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:35135
I20260925 12:25:27.114048 18690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:35135 every 8 connection(s)
I20260925 12:25:27.114423 18562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:27.114792 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18562
I20260925 12:25:27.114883 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:27.115114 1214 external_mini_cluster.cc:1428] Setting key dc9921b281f3e8b0c6b8cf783ae32e2a
Sep 25 12:25:27 dist-test-slave-l9ww krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:27.124765 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.115458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:40747 (local address 127.1.47.190:39955)
0925 12:25:27.115516 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.115518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.115804 (+ 286us) server_negotiation.cc:407] Connection header received
0925 12:25:27.116437 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.116438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.116448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.116495 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.117647 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.118168 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.119220 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.119413 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.119471 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.121782 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.121783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.121784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:27.121791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.123293 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:27.123733 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:27.123734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:27.123734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:27.123803 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:27.124142 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:27.124143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:27.124143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:27.124284 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.124340 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.124632 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.124699 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:25:27.124908 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.115408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:39955 (local address 127.1.47.131:40747)
0925 12:25:27.115632 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.115676 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.116345 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.116637 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.116641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.117310 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:27.117529 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.117534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.118282 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.118283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.119136 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.119138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.119489 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.119490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.119596 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.120173 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:27.120181 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:27.121653 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:27.123388 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:27.123396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:27.123398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:27.123653 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:27.123963 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:27.123963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:27.123964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:27.124074 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:27.124348 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:27.124352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:27.124475 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.124555 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.124686 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":121,"threads_started":1}
I20260925 12:25:27.125499 18691 heartbeater.cc:347] Connected to a master server at 127.1.47.190:39955
I20260925 12:25:27.125588 18691 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:27.125803 18691 heartbeater.cc:511] Master 127.1.47.190:39955 requested a full tablet report, sending...
I20260925 12:25:27.126194 18232 ts_manager.cc:194] Registered new tserver with Master: f840db609a984b8fa9f96c7ba42cd85e (127.1.47.131:35135)
I20260925 12:25:27.126700 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:40747
I20260925 12:25:27.126847 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:27.134881 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.128542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41528 (local address 127.1.47.190:39955)
0925 12:25:27.128595 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.128597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.128723 (+ 126us) server_negotiation.cc:407] Connection header received
0925 12:25:27.128780 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.128781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.128790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.128846 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.129368 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.129918 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.130649 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.130826 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.130884 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.132296 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.132298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.132298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:27.132305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.133591 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:27.133964 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:27.133966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:27.133966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:27.134030 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:27.134295 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:27.134296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:27.134296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:27.134463 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.134562 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.134739 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.134813 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 12:25:27.136734 18232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:27.138370 18232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:27.145185 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46701 (local address 127.0.0.1:40918)
0925 12:25:27.141482 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.141525 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.141592 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.141938 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.141941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.141944 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.142015 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.142021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143119 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.143120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.144149 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.144152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.144851 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.144852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.144930 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.144943 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.145046 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.145103 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":108,"threads_started":1}
I20260925 12:25:27.145195 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.1.47.130:46701)
0925 12:25:27.141750 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.141754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.141763 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:27.141808 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.141809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.141847 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.141924 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.142099 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143019 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.144236 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.144748 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.144847 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.144911 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.145064 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.145104 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":88,"threads_started":1}
I20260925 12:25:27.146035 18491 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 (DEFAULT_TABLE table=test-table [id=7367d0b59dc8426bb53e2a2c4e3c1e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.146273 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:35135 (local address 127.0.0.1:58688)
0925 12:25:27.142025 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.142039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.142104 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.142346 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.142347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.142349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.142398 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.142399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143729 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.143730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.145042 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.145043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.146021 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.146022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.146094 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.146099 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.146145 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.146192 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":86,"thread_start_us":67,"threads_started":1}
I20260925 12:25:27.146435 18491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.146474 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.1.47.129:34485)
0925 12:25:27.142240 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.142245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.142254 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:27.142316 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.142319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.142335 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.142405 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.142526 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143605 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.145143 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.145966 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.146026 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.146075 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.146331 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.146381 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":81,"threads_started":1}
I20260925 12:25:27.146603 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.1.47.131:35135)
0925 12:25:27.141657 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.141662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.142103 (+ 441us) server_negotiation.cc:407] Connection header received
0925 12:25:27.142171 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.142173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.142189 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.142265 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.142476 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143626 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.145143 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.145860 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.145930 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.145986 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.146474 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.146516 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260925 12:25:27.146723 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.141149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34485 (local address 127.0.0.1:47038)
0925 12:25:27.141481 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.141525 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.141592 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.142424 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.142424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.142426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.142468 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.142469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.143729 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.143730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.145041 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.145043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.146103 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.146104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.146162 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.146164 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.146206 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.146237 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":66,"thread_start_us":58,"threads_started":1}
I20260925 12:25:27.147307 18625 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 (DEFAULT_TABLE table=test-table [id=7367d0b59dc8426bb53e2a2c4e3c1e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.147373 18357 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 (DEFAULT_TABLE table=test-table [id=7367d0b59dc8426bb53e2a2c4e3c1e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.147639 18625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.147711 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.149986 18708 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Bootstrap starting.
I20260925 12:25:27.150691 18708 tablet_bootstrap.cc:654] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.150696 18709 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Bootstrap starting.
I20260925 12:25:27.151168 18710 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Bootstrap starting.
I20260925 12:25:27.151326 18708 log.cc:824] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.151549 18709 tablet_bootstrap.cc:654] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.151857 18710 tablet_bootstrap.cc:654] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.152134 18709 log.cc:824] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.152444 18710 log.cc:824] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.152618 18708 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: No bootstrap required, opened a new log
I20260925 12:25:27.152688 18708 ts_tablet_manager.cc:1397] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:25:27.152920 18709 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: No bootstrap required, opened a new log
I20260925 12:25:27.153002 18709 ts_tablet_manager.cc:1397] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260925 12:25:27.154038 18708 raft_consensus.cc:348] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.154189 18708 raft_consensus.cc:374] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.154212 18708 raft_consensus.cc:729] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c54555893c41c5a07f055270e3964d, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.154347 18708 consensus_queue.cc:260] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.154436 18709 raft_consensus.cc:348] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.154636 18709 raft_consensus.cc:374] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.154663 18708 ts_tablet_manager.cc:1428] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:25:27.154697 18709 raft_consensus.cc:729] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c898a9e79a4038b8039abbd51a62fd, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.154709 18557 heartbeater.cc:503] Master 127.1.47.190:39955 was elected leader, sending a full tablet report...
I20260925 12:25:27.154809 18709 consensus_queue.cc:260] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.155112 18709 ts_tablet_manager.cc:1428] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 12:25:27.155128 18423 heartbeater.cc:503] Master 127.1.47.190:39955 was elected leader, sending a full tablet report...
I20260925 12:25:27.155371 18710 tablet_bootstrap.cc:492] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: No bootstrap required, opened a new log
I20260925 12:25:27.155433 18710 ts_tablet_manager.cc:1397] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260925 12:25:27.157353 18710 raft_consensus.cc:348] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.157485 18710 raft_consensus.cc:374] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.157505 18710 raft_consensus.cc:729] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f840db609a984b8fa9f96c7ba42cd85e, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.157608 18710 consensus_queue.cc:260] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.157851 18710 ts_tablet_manager.cc:1428] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260925 12:25:27.157924 18691 heartbeater.cc:503] Master 127.1.47.190:39955 was elected leader, sending a full tablet report...
W20260925 12:25:27.194581 18558 tablet.cc:2369] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:27.263198 18716 raft_consensus.cc:482] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:27.263329 18716 raft_consensus.cc:504] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.263756 18716 leader_election.cc:288] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22c54555893c41c5a07f055270e3964d (127.1.47.130:46701), 55c898a9e79a4038b8039abbd51a62fd (127.1.47.129:34485)
I20260925 12:25:27.265336 18715 raft_consensus.cc:482] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:27.265486 18715 raft_consensus.cc:504] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.265887 18715 leader_election.cc:288] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22c54555893c41c5a07f055270e3964d (127.1.47.130:46701), f840db609a984b8fa9f96c7ba42cd85e (127.1.47.131:35135)
I20260925 12:25:27.267764 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.264082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:47935 (local address 127.1.47.129:34485)
0925 12:25:27.264137 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.264140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.264253 (+ 113us) server_negotiation.cc:407] Connection header received
0925 12:25:27.264326 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.264327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.264347 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.264400 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.264561 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.265269 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.266867 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267418 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.267461 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.267469 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.267676 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.267710 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 12:25:27.267764 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.263993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46701 (local address 127.1.47.131:57217)
0925 12:25:27.264053 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.264068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.264116 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.264319 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.264320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.264322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.264407 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.264409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.265375 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.265375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.266661 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.266665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267503 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.267503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.267593 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.267599 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.267646 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.267690 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":68}
I20260925 12:25:27.268020 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.263993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34485 (local address 127.1.47.131:47935)
0925 12:25:27.264168 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.264179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.264248 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.264414 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.264414 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.264416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.264478 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.264480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.265362 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.265363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.266779 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.266781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267523 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.267523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.267593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.267600 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.267905 (+ 305us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.267952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":66,"thread_start_us":100,"threads_started":1}
I20260925 12:25:27.268076 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.264095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:57217 (local address 127.1.47.130:46701)
0925 12:25:27.264140 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.264142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.264151 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:27.264182 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.264183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.264201 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.264292 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.264491 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.265253 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.266762 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267391 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.267461 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.267469 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.267961 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.268000 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":24}
I20260925 12:25:27.268317 18511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "f840db609a984b8fa9f96c7ba42cd85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c54555893c41c5a07f055270e3964d" is_pre_election: true
I20260925 12:25:27.268319 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "f840db609a984b8fa9f96c7ba42cd85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c898a9e79a4038b8039abbd51a62fd" is_pre_election: true
I20260925 12:25:27.268440 18377 raft_consensus.cc:2454] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f840db609a984b8fa9f96c7ba42cd85e in term 0.
I20260925 12:25:27.268463 18511 raft_consensus.cc:2454] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f840db609a984b8fa9f96c7ba42cd85e in term 0.
I20260925 12:25:27.268662 18579 leader_election.cc:302] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c898a9e79a4038b8039abbd51a62fd, f840db609a984b8fa9f96c7ba42cd85e; no voters:
I20260925 12:25:27.268802 18716 raft_consensus.cc:2781] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:27.268857 18716 raft_consensus.cc:482] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:27.268903 18716 raft_consensus.cc:3037] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.269804 18716 raft_consensus.cc:504] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.270095 18716 leader_election.cc:288] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [CANDIDATE]: Term 1 election: Requested vote from peers 22c54555893c41c5a07f055270e3964d (127.1.47.130:46701), 55c898a9e79a4038b8039abbd51a62fd (127.1.47.129:34485)
I20260925 12:25:27.270102 18511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "f840db609a984b8fa9f96c7ba42cd85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c54555893c41c5a07f055270e3964d"
I20260925 12:25:27.270169 18511 raft_consensus.cc:3037] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.270192 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.266001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:46701 (local address 127.1.47.129:43749)
0925 12:25:27.266051 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.266062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.266101 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.266647 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.266648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.266649 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.266701 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.266702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267764 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.267766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.269125 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.269127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.269933 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.269934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.270016 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.270021 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.270082 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.270123 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:25:27.270300 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "f840db609a984b8fa9f96c7ba42cd85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c898a9e79a4038b8039abbd51a62fd"
I20260925 12:25:27.270370 18377 raft_consensus.cc:3037] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.270710 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.266342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:35391 (local address 127.1.47.131:35135)
0925 12:25:27.266384 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.266386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.266395 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:27.266445 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.266446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.266457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.266491 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.266755 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267589 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.269669 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.270353 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.270419 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.270427 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.270596 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.270631 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:27.270963 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.266089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:35135 (local address 127.1.47.129:35391)
0925 12:25:27.266288 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.266299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.266383 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.266561 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.266562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.266565 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.266651 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.266652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.268379 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.268380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.269578 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.269580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.270418 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.270419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.270499 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.270504 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.270833 (+ 329us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.270876 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260925 12:25:27.270962 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.266229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:43749 (local address 127.1.47.130:46701)
0925 12:25:27.266503 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.266505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.266513 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:25:27.266535 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.266536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.266546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.266773 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.266809 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.267663 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.269236 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.269843 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.270001 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.270012 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.270695 (+ 683us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.270743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":251,"thread_start_us":103,"threads_started":1}
I20260925 12:25:27.271245 18377 raft_consensus.cc:2454] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f840db609a984b8fa9f96c7ba42cd85e in term 1.
I20260925 12:25:27.271426 18579 leader_election.cc:302] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c898a9e79a4038b8039abbd51a62fd, f840db609a984b8fa9f96c7ba42cd85e; no voters:
I20260925 12:25:27.271389 18645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "55c898a9e79a4038b8039abbd51a62fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f840db609a984b8fa9f96c7ba42cd85e" is_pre_election: true
I20260925 12:25:27.271528 18645 raft_consensus.cc:2379] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55c898a9e79a4038b8039abbd51a62fd in current term 1: Already voted for candidate f840db609a984b8fa9f96c7ba42cd85e in this term.
I20260925 12:25:27.271554 18510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4f5eab77b14ab3bc36d55fa7faa3f9" candidate_uuid: "55c898a9e79a4038b8039abbd51a62fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c54555893c41c5a07f055270e3964d" is_pre_election: true
I20260925 12:25:27.271610 18716 raft_consensus.cc:2781] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:27.271100 18511 raft_consensus.cc:2454] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f840db609a984b8fa9f96c7ba42cd85e in term 1.
I20260925 12:25:27.271776 18716 raft_consensus.cc:686] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 LEADER]: Becoming Leader. State: Replica: f840db609a984b8fa9f96c7ba42cd85e, State: Running, Role: LEADER
I20260925 12:25:27.271901 18716 consensus_queue.cc:237] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } }
I20260925 12:25:27.271957 18310 leader_election.cc:302] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55c898a9e79a4038b8039abbd51a62fd; no voters: 22c54555893c41c5a07f055270e3964d, f840db609a984b8fa9f96c7ba42cd85e
I20260925 12:25:27.272166 18715 raft_consensus.cc:2726] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 12:25:27.272715 18231 catalog_manager.cc:5495] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e reported cstate change: term changed from 0 to 1, leader changed from <none> to f840db609a984b8fa9f96c7ba42cd85e (127.1.47.131). New cstate: current_term: 1 leader_uuid: "f840db609a984b8fa9f96c7ba42cd85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f840db609a984b8fa9f96c7ba42cd85e" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 35135 } health_report { overall_health: HEALTHY } } }
W20260925 12:25:27.292520 18424 tablet.cc:2369] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:27.293183 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.289345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.1.47.131:35135)
0925 12:25:27.289407 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.289409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.289577 (+ 168us) server_negotiation.cc:407] Connection header received
0925 12:25:27.289679 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.289680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.289706 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.289809 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:27.290029 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.290775 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.292034 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.292492 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.292567 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.292775 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:27.292922 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:27.293003 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.293087 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.293108 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 12:25:27.295837 18511 raft_consensus.cc:1261] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 1 FOLLOWER]: Refusing update from remote peer f840db609a984b8fa9f96c7ba42cd85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:27.296087 18377 raft_consensus.cc:1261] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 1 FOLLOWER]: Refusing update from remote peer f840db609a984b8fa9f96c7ba42cd85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:27.296271 18720 consensus_queue.cc:1035] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22c54555893c41c5a07f055270e3964d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 46701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:27.296398 18716 consensus_queue.cc:1035] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c898a9e79a4038b8039abbd51a62fd" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:27.299162 18723 mvcc.cc:205] Tried to move back new op lower bound from 7333229065400729600 to 7333229065306345472. Current Snapshot: MvccSnapshot[applied={T|T < 7333229065400729600}]
I20260925 12:25:27.299916 18725 mvcc.cc:205] Tried to move back new op lower bound from 7333229065400729600 to 7333229065306345472. Current Snapshot: MvccSnapshot[applied={T|T < 7333229065400729600}]
I20260925 12:25:27.302570 18232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:27.302687 18232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:27.303402 18232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8de2ab63ff846ed858a504d3628faab: Sending DeleteTablet for 3 replicas of tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9
I20260925 12:25:27.303710 18491 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:27 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40918
I20260925 12:25:27.303835 18357 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:27 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:47038
I20260925 12:25:27.303921 18625 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:27 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:58688
I20260925 12:25:27.303964 18733 tablet_replica.cc:330] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: stopping tablet replica
I20260925 12:25:27.304070 18733 raft_consensus.cc:2229] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:27.304189 18733 raft_consensus.cc:2258] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:27.304185 18734 tablet_replica.cc:330] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd: stopping tablet replica
I20260925 12:25:27.304286 18734 raft_consensus.cc:2229] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 55c898a9e79a4038b8039abbd51a62fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:27.304327 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18294
I20260925 12:25:27.304595 18733 ts_tablet_manager.cc:1905] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:27.306833 18733 ts_tablet_manager.cc:1918] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:27.306902 18733 log.cc:1196] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/5d4f5eab77b14ab3bc36d55fa7faa3f9
I20260925 12:25:27.307168 18733 ts_tablet_manager.cc:1939] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P 22c54555893c41c5a07f055270e3964d: Deleting consensus metadata
I20260925 12:25:27.307560 18216 catalog_manager.cc:4841] TS 22c54555893c41c5a07f055270e3964d (127.1.47.130:46701): tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 (table test-table [id=7367d0b59dc8426bb53e2a2c4e3c1e0c]) successfully deleted
W20260925 12:25:27.311793 18217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:34485 (error 108)
I20260925 12:25:27.312404 18735 tablet_replica.cc:330] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: stopping tablet replica
I20260925 12:25:27.312418 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.312264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:34485 (local address 127.0.0.1:47052)
0925 12:25:27.312334 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.312380 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 12:25:27.312422 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18428
I20260925 12:25:27.312542 18735 raft_consensus.cc:2229] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:25:27.312693 18735 raft_consensus.cc:2258] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:27.313392 18735 ts_tablet_manager.cc:1905] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:25:27.313467 18217 catalog_manager.cc:4584] TS 55c898a9e79a4038b8039abbd51a62fd (127.1.47.129:34485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9: Network error: Client connection negotiation failed: client connection to 127.1.47.129:34485: connect: Connection refused (error 111)
I20260925 12:25:27.315518 18735 ts_tablet_manager.cc:1918] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:27.315619 18735 log.cc:1196] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/5d4f5eab77b14ab3bc36d55fa7faa3f9
I20260925 12:25:27.315936 18735 ts_tablet_manager.cc:1939] T 5d4f5eab77b14ab3bc36d55fa7faa3f9 P f840db609a984b8fa9f96c7ba42cd85e: Deleting consensus metadata
I20260925 12:25:27.316370 18217 catalog_manager.cc:4841] TS f840db609a984b8fa9f96c7ba42cd85e (127.1.47.131:35135): tablet 5d4f5eab77b14ab3bc36d55fa7faa3f9 (table test-table [id=7367d0b59dc8426bb53e2a2c4e3c1e0c]) successfully deleted
I20260925 12:25:27.320148 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18562
I20260925 12:25:27.327128 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18200
2026-09-25T12:25:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2046 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T12:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:27Z Disabled control of system clock
I20260925 12:25:27.357293 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36465
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:35653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:27.438185 18740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:27.438416 18740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:27.438438 18740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:27.439653 18740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:27.439698 18740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:27.439711 18740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:27.439723 18740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:27.439782 18740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:27.441201 18740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36465
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:27.441486 18740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:27.441817 18740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:27.446671 18745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.446671 18746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.446895 18748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:27.447371 18740 server_base.cc:1034] running on GCE node
I20260925 12:25:27.447671 18740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:27.448068 18740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:27.449299 18740 hybrid_clock.cc:648] HybridClock initialized: now 1790339127449288 us; error 38 us; skew 500 ppm
I20260925 12:25:27.450695 18740 webserver.cc:460] Webserver started at http://127.1.47.190:41375/ using document root <none> and password file <none>
I20260925 12:25:27.450980 18740 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:27.451058 18740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:27.451215 18740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:27.452422 18740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "30a26d63386442ffb24d76e106f63842"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "bda29b0fb47aec99002623bbd0b618d2"
server_key_iv: "134fdae880377835a8bca2a53831ce95"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.452802 18740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "30a26d63386442ffb24d76e106f63842"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "bda29b0fb47aec99002623bbd0b618d2"
server_key_iv: "134fdae880377835a8bca2a53831ce95"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.454221 18740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:27.455075 18754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.455334 18740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:27.455471 18740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "30a26d63386442ffb24d76e106f63842"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "bda29b0fb47aec99002623bbd0b618d2"
server_key_iv: "134fdae880377835a8bca2a53831ce95"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.455577 18740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:27.471757 18740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:27.472193 18740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:27.472371 18740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:27.476365 18740 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:36465
I20260925 12:25:27.476404 18806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:36465 every 8 connection(s)
I20260925 12:25:27.476764 18740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:27.477818 18807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.480185 18807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: Bootstrap starting.
I20260925 12:25:27.480731 18807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.480986 18807 log.cc:824] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.481679 18807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: No bootstrap required, opened a new log
I20260925 12:25:27.482260 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18740
I20260925 12:25:27.482391 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:27.482595 1214 external_mini_cluster.cc:1428] Setting key 9788b1259e50c6b32a0c0991fa9c32f8
I20260925 12:25:27.483697 18807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } }
I20260925 12:25:27.483841 18807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.483940 18807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a26d63386442ffb24d76e106f63842, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.484086 18807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } }
I20260925 12:25:27.484153 18807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:27.484175 18807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:27.484258 18807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.484810 18807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } }
I20260925 12:25:27.484943 18807 leader_election.cc:302] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30a26d63386442ffb24d76e106f63842; no voters:
I20260925 12:25:27.485143 18807 leader_election.cc:288] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:27.485178 18812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:27.485287 18812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [term 1 LEADER]: Becoming Leader. State: Replica: 30a26d63386442ffb24d76e106f63842, State: Running, Role: LEADER
I20260925 12:25:27.485370 18812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } }
I20260925 12:25:27.485416 18807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:27.485723 18812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30a26d63386442ffb24d76e106f63842. Latest consensus state: current_term: 1 leader_uuid: "30a26d63386442ffb24d76e106f63842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } } }
I20260925 12:25:27.485798 18812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:27.485731 18814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30a26d63386442ffb24d76e106f63842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a26d63386442ffb24d76e106f63842" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36465 } } }
I20260925 12:25:27.485844 18814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:27.486927 18817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:27.487437 18817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:27.489300 18817 catalog_manager.cc:1347] Generated new cluster ID: 097fc0d8487941dda20fa7a3943734ce
I20260925 12:25:27.489351 18817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:27.489641 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.483160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.1.47.190:36465)
0925 12:25:27.483381 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.483389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.483413 (+ 24us) server_negotiation.cc:407] Connection header received
0925 12:25:27.483705 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.483714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.483794 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.484050 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.484486 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.486190 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.487327 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.487963 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.488132 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.488887 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.488891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.488892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:27.489104 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.489153 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:27.489168 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.489238 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.489345 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.489431 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260925 12:25:27.501715 18817 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:27.502262 18817 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:27.509071 18817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: Generated new TSK 0
I20260925 12:25:27.509352 18817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:27.511485 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--builtin_ntp_servers=127.1.47.148:35653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:27.595008 18831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:27.595273 18831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:27.595315 18831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:27.596729 18831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:27.596800 18831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:27.596845 18831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:27.598304 18831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:27.598593 18831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:27.598922 18831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:27.605124 18836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.605115 18837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.605115 18839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:27.605556 18831 server_base.cc:1034] running on GCE node
I20260925 12:25:27.605768 18831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:27.606024 18831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:27.607194 18831 hybrid_clock.cc:648] HybridClock initialized: now 1790339127607177 us; error 27 us; skew 500 ppm
I20260925 12:25:27.608757 18831 webserver.cc:460] Webserver started at http://127.1.47.129:44693/ using document root <none> and password file <none>
I20260925 12:25:27.609036 18831 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:27.609095 18831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:27.609236 18831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:27.610870 18831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "fd75abc7e27f478ea4be01825992c364"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "0276f525fea00630deb6cc2ccf0ce4cf"
server_key_iv: "f06338d5b55308dd72e3888b2c82eb70"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.611361 18831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "fd75abc7e27f478ea4be01825992c364"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "0276f525fea00630deb6cc2ccf0ce4cf"
server_key_iv: "f06338d5b55308dd72e3888b2c82eb70"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.613098 18831 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260925 12:25:27.613988 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.614192 18831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:27.614296 18831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "fd75abc7e27f478ea4be01825992c364"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "0276f525fea00630deb6cc2ccf0ce4cf"
server_key_iv: "f06338d5b55308dd72e3888b2c82eb70"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.614377 18831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:27.622494 18831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:27.622813 18831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:27.622972 18831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:27.623194 18831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:27.623525 18831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:27.623565 18831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.623588 18831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:27.623600 18831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.630003 18831 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:36191
I20260925 12:25:27.630120 18958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:36191 every 8 connection(s)
I20260925 12:25:27.630442 18831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:27.636085 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.631309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41891 (local address 127.1.47.190:36465)
0925 12:25:27.631385 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.631387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.631638 (+ 251us) server_negotiation.cc:407] Connection header received
0925 12:25:27.632236 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.632238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.632253 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.632317 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.633030 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.633559 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.634579 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.634785 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.634862 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.635738 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.635740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.635740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:27.635749 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.635764 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:27.635775 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.635837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.635970 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.636001 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 12:25:27.636477 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18831
I20260925 12:25:27.636554 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:27.636749 1214 external_mini_cluster.cc:1428] Setting key 285cdf0fd48a2c1af49ce606e526cee5
I20260925 12:25:27.636909 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.631289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36465 (local address 127.1.47.129:41891)
0925 12:25:27.631510 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.631547 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.632127 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.632402 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.632406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.632414 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:27.632924 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.632929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.633658 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.633659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.634469 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.634471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.634871 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.634872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.635026 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.635581 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:27.635581 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:27.635611 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:27.635637 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:27.635835 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:27.635842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:27.635846 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.635974 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.636082 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
I20260925 12:25:27.637570 18959 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36465
I20260925 12:25:27.637691 18959 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:27.637876 18959 heartbeater.cc:511] Master 127.1.47.190:36465 requested a full tablet report, sending...
I20260925 12:25:27.637998 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--builtin_ntp_servers=127.1.47.148:35653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 12:25:27.638474 18771 ts_manager.cc:194] Registered new tserver with Master: fd75abc7e27f478ea4be01825992c364 (127.1.47.129:36191)
I20260925 12:25:27.639528 18771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.129:41891
W20260925 12:25:27.721416 18962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:27.721637 18962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:27.721659 18962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:27.722826 18962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:27.722868 18962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:27.722895 18962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:27.724385 18962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:27.724644 18962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:27.724953 18962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:27.729976 18967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.729985 18970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.729955 18968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:27.730283 18962 server_base.cc:1034] running on GCE node
I20260925 12:25:27.730465 18962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:27.730671 18962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:27.731945 18962 hybrid_clock.cc:648] HybridClock initialized: now 1790339127731924 us; error 32 us; skew 500 ppm
I20260925 12:25:27.733469 18962 webserver.cc:460] Webserver started at http://127.1.47.130:42947/ using document root <none> and password file <none>
I20260925 12:25:27.733870 18962 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:27.733952 18962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:27.734122 18962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:27.735363 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "da8d6af780d24b1d801768eaed6a2962"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "5b4ec507fc4e52f0f5ef614935784f1c"
server_key_iv: "64d951d0643fef34f8e25623fb2caa21"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.735783 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "da8d6af780d24b1d801768eaed6a2962"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "5b4ec507fc4e52f0f5ef614935784f1c"
server_key_iv: "64d951d0643fef34f8e25623fb2caa21"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.737430 18962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:27.738248 18976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.738445 18962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:27.738547 18962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "da8d6af780d24b1d801768eaed6a2962"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "5b4ec507fc4e52f0f5ef614935784f1c"
server_key_iv: "64d951d0643fef34f8e25623fb2caa21"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.738622 18962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:27.754354 18962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:27.754730 18962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:27.754910 18962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:27.755164 18962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:27.755528 18962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:27.755584 18962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.755630 18962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:27.755656 18962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.761860 18962 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:44553
I20260925 12:25:27.761987 19089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:44553 every 8 connection(s)
I20260925 12:25:27.762285 18962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:25:27.763409 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 18962
I20260925 12:25:27.763514 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:27.763711 1214 external_mini_cluster.cc:1428] Setting key 7164ef2dd66478dadfc54b631f526536
I20260925 12:25:27.764703 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--builtin_ntp_servers=127.1.47.148:35653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 12:25:27.769125 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.763095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:33491 (local address 127.1.47.190:36465)
0925 12:25:27.763189 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.763191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.763540 (+ 349us) server_negotiation.cc:407] Connection header received
0925 12:25:27.764238 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.764239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.764251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.764320 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.765139 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.765920 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.767531 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.767716 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.767772 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.768855 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.768857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.768857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:27.768861 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.768875 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:27.768879 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.768921 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.769039 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.769066 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 12:25:27.769521 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.763061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36465 (local address 127.1.47.130:33491)
0925 12:25:27.763348 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.763417 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.764135 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.764416 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.764420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.764430 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:27.765047 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.765052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.766048 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.766049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.767440 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.767443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.767808 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.767809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.768001 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.768676 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:27.768677 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:27.768720 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:27.768754 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:27.768946 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:27.768952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:27.768956 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.769039 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.769162 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260925 12:25:27.770079 19090 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36465
I20260925 12:25:27.770187 19090 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:27.770408 19090 heartbeater.cc:511] Master 127.1.47.190:36465 requested a full tablet report, sending...
I20260925 12:25:27.770813 18771 ts_manager.cc:194] Registered new tserver with Master: da8d6af780d24b1d801768eaed6a2962 (127.1.47.130:44553)
I20260925 12:25:27.771190 18771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.130:33491
W20260925 12:25:27.847919 19093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:27.848132 19093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:27.848240 19093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:27.849445 19093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:27.849486 19093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:27.849512 19093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:27.850885 19093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:35653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:27.851158 19093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:27.851477 19093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:27.856580 19099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.856616 19098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:27.856580 19101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:27.856964 19093 server_base.cc:1034] running on GCE node
I20260925 12:25:27.857146 19093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:27.857398 19093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:27.858574 19093 hybrid_clock.cc:648] HybridClock initialized: now 1790339127858562 us; error 40 us; skew 500 ppm
I20260925 12:25:27.860103 19093 webserver.cc:460] Webserver started at http://127.1.47.131:39001/ using document root <none> and password file <none>
I20260925 12:25:27.860368 19093 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:27.860417 19093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:27.860574 19093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:27.861688 19093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "f18236bfa0474699b1853b88d7c8e75b"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "1f81399d80c66532516b03498291b4a6"
server_key_iv: "8ba269fbb34d437ec723ad71417b456b"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.862071 19093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "f18236bfa0474699b1853b88d7c8e75b"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "1f81399d80c66532516b03498291b4a6"
server_key_iv: "8ba269fbb34d437ec723ad71417b456b"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.863605 19093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:27.864498 19107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.864717 19093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 12:25:27.864811 19093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "f18236bfa0474699b1853b88d7c8e75b"
format_stamp: "Formatted at 2026-09-25 12:25:27 on dist-test-slave-l9ww"
server_key: "1f81399d80c66532516b03498291b4a6"
server_key_iv: "8ba269fbb34d437ec723ad71417b456b"
server_key_version: "encryptionkey@0"
I20260925 12:25:27.864869 19093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:27.873324 19093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:27.873584 19093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:27.873685 19093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:27.873855 19093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:27.874106 19093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:27.874135 19093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.874156 19093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:27.874168 19093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:27.879931 19093 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:38251
I20260925 12:25:27.879969 19220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:38251 every 8 connection(s)
I20260925 12:25:27.880327 19093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:27.885658 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.881093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:48465 (local address 127.1.47.190:36465)
0925 12:25:27.881165 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.881167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.881451 (+ 284us) server_negotiation.cc:407] Connection header received
0925 12:25:27.881925 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.881927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.881938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.882003 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.882695 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.883184 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.884216 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.884411 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.884466 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.885320 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.885321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.885321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:27.885325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.885351 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:27.885355 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.885422 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.885525 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.885551 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:25:27.885790 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.881078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36465 (local address 127.1.47.131:48465)
0925 12:25:27.881324 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.881358 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.881837 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.882082 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.882085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.882093 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:27.882620 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.882624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.883278 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.883279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.884142 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.884145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.884485 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.884486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.884594 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.885189 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:27.885190 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:27.885219 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:27.885244 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:27.885437 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:27.885443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:27.885447 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.885513 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.885597 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":122,"threads_started":1}
I20260925 12:25:27.886480 19221 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36465
I20260925 12:25:27.886580 19221 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:27.886777 19221 heartbeater.cc:511] Master 127.1.47.190:36465 requested a full tablet report, sending...
I20260925 12:25:27.887213 18771 ts_manager.cc:194] Registered new tserver with Master: f18236bfa0474699b1853b88d7c8e75b (127.1.47.131:38251)
I20260925 12:25:27.887602 18771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.131:48465
I20260925 12:25:27.890228 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19093
I20260925 12:25:27.890305 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:27.890501 1214 external_mini_cluster.cc:1428] Setting key 35ab13b7aaec4f187b412963a8bb9e8c
I20260925 12:25:27.891644 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:27.896216 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.893170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.1.47.190:36465)
0925 12:25:27.893244 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.893246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.893357 (+ 111us) server_negotiation.cc:407] Connection header received
0925 12:25:27.893436 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.893437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.893450 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.893505 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:27.893679 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.894156 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.894937 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.895142 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.895230 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.895951 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:27.895952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:27.895952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:27.895957 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:27.895971 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:27.895976 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:27.896044 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.896104 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.896128 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:25:27.898025 18771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:27.899724 18771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:27.906360 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.1.47.131:38251)
0925 12:25:27.902389 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.902395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.902586 (+ 191us) server_negotiation.cc:407] Connection header received
0925 12:25:27.902633 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.902635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.902650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.902721 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.902896 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.903677 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.905150 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.905950 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.906044 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.906146 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.906231 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.906270 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260925 12:25:27.906349 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36191 (local address 127.0.0.1:50214)
0925 12:25:27.902210 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.902248 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.902318 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.902645 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.902648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.902650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.902725 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.902727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.904272 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.904273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.905277 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.905279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.906058 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.906059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.906139 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.906144 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.906187 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.906236 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260925 12:25:27.906560 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.1.47.129:36191)
0925 12:25:27.902456 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.902459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.902470 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:25:27.902520 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.902522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.902542 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.902646 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.902853 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.904172 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.905360 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.905957 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.906316 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.906416 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.906443 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.906484 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260925 12:25:27.906764 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:38251 (local address 127.0.0.1:33576)
0925 12:25:27.902465 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.902474 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.902524 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.902787 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.902788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.902790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.902821 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.902822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.903767 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.903769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.905053 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.905056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.906042 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.906043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.906126 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.906140 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.906219 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.906264 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":419,"threads_started":1}
I20260925 12:25:27.907241 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:44553 (local address 127.0.0.1:36012)
0925 12:25:27.902557 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.902570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.902633 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.902997 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.902998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.902999 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.903029 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.903030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.904421 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.904422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.905686 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.905698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.907015 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.907016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.907081 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.907085 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.907142 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.907180 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"spinlock_wait_cycles":384,"thread_start_us":358,"threads_started":1}
I20260925 12:25:27.907318 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.902228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36012 (local address 127.1.47.130:44553)
0925 12:25:27.902562 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.902567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.902792 (+ 225us) server_negotiation.cc:407] Connection header received
0925 12:25:27.902833 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.902835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.902851 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.902938 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.903117 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.904331 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.905801 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.906796 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.906890 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.906980 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.907188 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.907235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":88,"threads_started":1}
I20260925 12:25:27.907586 19155 tablet_service.cc:1467] Processing CreateTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c (DEFAULT_TABLE table=test-table [id=8b527728f450458ba9a170e8717adefe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.907582 18893 tablet_service.cc:1467] Processing CreateTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c (DEFAULT_TABLE table=test-table [id=8b527728f450458ba9a170e8717adefe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.907949 19155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9413b9bdea2e4143b3c87b28cfca303c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.907971 18893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9413b9bdea2e4143b3c87b28cfca303c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.908603 19024 tablet_service.cc:1467] Processing CreateTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c (DEFAULT_TABLE table=test-table [id=8b527728f450458ba9a170e8717adefe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:27.908921 19024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9413b9bdea2e4143b3c87b28cfca303c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:27.911046 19235 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: Bootstrap starting.
I20260925 12:25:27.911609 19236 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Bootstrap starting.
I20260925 12:25:27.911937 19235 tablet_bootstrap.cc:654] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.912279 19237 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Bootstrap starting.
I20260925 12:25:27.912307 19235 log.cc:824] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.912352 19236 tablet_bootstrap.cc:654] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.912678 19236 log.cc:824] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.912921 19237 tablet_bootstrap.cc:654] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:27.913215 19235 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: No bootstrap required, opened a new log
I20260925 12:25:27.913232 19237 log.cc:824] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:27.913292 19235 ts_tablet_manager.cc:1397] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:25:27.913517 19236 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: No bootstrap required, opened a new log
I20260925 12:25:27.913604 19236 ts_tablet_manager.cc:1397] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:25:27.914100 19237 tablet_bootstrap.cc:492] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: No bootstrap required, opened a new log
I20260925 12:25:27.914184 19237 ts_tablet_manager.cc:1397] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:25:27.915696 19236 raft_consensus.cc:348] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.915870 19236 raft_consensus.cc:374] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.915959 19236 raft_consensus.cc:729] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f18236bfa0474699b1853b88d7c8e75b, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.916091 19236 consensus_queue.cc:260] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.916394 19236 ts_tablet_manager.cc:1428] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:25:27.916540 19221 heartbeater.cc:503] Master 127.1.47.190:36465 was elected leader, sending a full tablet report...
I20260925 12:25:27.916438 19235 raft_consensus.cc:348] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.916620 19235 raft_consensus.cc:374] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.916659 19235 raft_consensus.cc:729] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75abc7e27f478ea4be01825992c364, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.916770 19235 consensus_queue.cc:260] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.916882 19237 raft_consensus.cc:348] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.917029 19235 ts_tablet_manager.cc:1428] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260925 12:25:27.917032 19237 raft_consensus.cc:374] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:27.917064 19237 raft_consensus.cc:729] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8d6af780d24b1d801768eaed6a2962, State: Initialized, Role: FOLLOWER
I20260925 12:25:27.917136 18959 heartbeater.cc:503] Master 127.1.47.190:36465 was elected leader, sending a full tablet report...
I20260925 12:25:27.917179 19237 consensus_queue.cc:260] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.917435 19237 ts_tablet_manager.cc:1428] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 12:25:27.917544 19090 heartbeater.cc:503] Master 127.1.47.190:36465 was elected leader, sending a full tablet report...
I20260925 12:25:27.970750 19241 raft_consensus.cc:482] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:27.970889 19241 raft_consensus.cc:504] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.971246 19241 leader_election.cc:288] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75abc7e27f478ea4be01825992c364 (127.1.47.129:36191), da8d6af780d24b1d801768eaed6a2962 (127.1.47.130:44553)
I20260925 12:25:27.974218 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.971576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:33255 (local address 127.1.47.130:44553)
0925 12:25:27.971646 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.971648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.971754 (+ 106us) server_negotiation.cc:407] Connection header received
0925 12:25:27.971793 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.971794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.971803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.971896 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.972055 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.972531 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.973474 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.973862 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.973926 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.973933 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.974097 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.974145 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:25:27.974236 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.971468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:44553 (local address 127.1.47.131:33255)
0925 12:25:27.971652 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.971664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.971745 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.971905 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.971906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.971908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.971980 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.971982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.972623 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.972623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.973388 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.973390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.973948 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.973949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.974021 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.974025 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.974079 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.974108 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260925 12:25:27.974318 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.971543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:39809 (local address 127.1.47.129:36191)
0925 12:25:27.971594 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.971597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.971640 (+ 43us) server_negotiation.cc:407] Connection header received
0925 12:25:27.971675 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.971676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.971687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.971750 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:27.971952 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.972476 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.973525 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.973896 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.973990 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.973997 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.974195 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.974243 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 12:25:27.974349 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.971452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36191 (local address 127.1.47.131:39809)
0925 12:25:27.971517 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:27.971531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:27.971577 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:27.971777 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:27.971778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:27.971780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:27.971886 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.971888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.972565 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.972566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.973411 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:27.973414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.974012 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.974013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:27.974104 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.974107 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:25:27.974163 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 12:25:27.974202 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260925 12:25:27.974650 18913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9413b9bdea2e4143b3c87b28cfca303c" candidate_uuid: "f18236bfa0474699b1853b88d7c8e75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75abc7e27f478ea4be01825992c364" is_pre_election: true
I20260925 12:25:27.974651 19044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9413b9bdea2e4143b3c87b28cfca303c" candidate_uuid: "f18236bfa0474699b1853b88d7c8e75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8d6af780d24b1d801768eaed6a2962" is_pre_election: true
I20260925 12:25:27.974820 19044 raft_consensus.cc:2454] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f18236bfa0474699b1853b88d7c8e75b in term 0.
I20260925 12:25:27.974820 18913 raft_consensus.cc:2454] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f18236bfa0474699b1853b88d7c8e75b in term 0.
I20260925 12:25:27.975142 19108 leader_election.cc:302] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8d6af780d24b1d801768eaed6a2962, f18236bfa0474699b1853b88d7c8e75b; no voters:
I20260925 12:25:27.975340 19241 raft_consensus.cc:2781] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:27.975384 19241 raft_consensus.cc:482] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:27.975400 19241 raft_consensus.cc:3037] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.976262 19241 raft_consensus.cc:504] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.976423 19241 leader_election.cc:288] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [CANDIDATE]: Term 1 election: Requested vote from peers fd75abc7e27f478ea4be01825992c364 (127.1.47.129:36191), da8d6af780d24b1d801768eaed6a2962 (127.1.47.130:44553)
I20260925 12:25:27.976603 19044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9413b9bdea2e4143b3c87b28cfca303c" candidate_uuid: "f18236bfa0474699b1853b88d7c8e75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8d6af780d24b1d801768eaed6a2962"
I20260925 12:25:27.976653 18913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9413b9bdea2e4143b3c87b28cfca303c" candidate_uuid: "f18236bfa0474699b1853b88d7c8e75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75abc7e27f478ea4be01825992c364"
I20260925 12:25:27.976693 19044 raft_consensus.cc:3037] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.976792 18913 raft_consensus.cc:3037] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:27.977501 19044 raft_consensus.cc:2454] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f18236bfa0474699b1853b88d7c8e75b in term 1.
I20260925 12:25:27.977665 19108 leader_election.cc:302] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8d6af780d24b1d801768eaed6a2962, f18236bfa0474699b1853b88d7c8e75b; no voters:
I20260925 12:25:27.977689 18913 raft_consensus.cc:2454] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f18236bfa0474699b1853b88d7c8e75b in term 1.
I20260925 12:25:27.977748 19241 raft_consensus.cc:2781] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:27.977944 19241 raft_consensus.cc:686] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 1 LEADER]: Becoming Leader. State: Replica: f18236bfa0474699b1853b88d7c8e75b, State: Running, Role: LEADER
I20260925 12:25:27.978065 19241 consensus_queue.cc:237] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } }
I20260925 12:25:27.978947 18771 catalog_manager.cc:5495] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b reported cstate change: term changed from 0 to 1, leader changed from <none> to f18236bfa0474699b1853b88d7c8e75b (127.1.47.131). New cstate: current_term: 1 leader_uuid: "f18236bfa0474699b1853b88d7c8e75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f18236bfa0474699b1853b88d7c8e75b" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 38251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 12:25:27.999588 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:27.997118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.1.47.131:38251)
0925 12:25:27.997162 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:27.997165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:27.997174 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:27.997269 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:27.997270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:27.997290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:27.997383 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:27.997496 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.997965 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.998761 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:27.998957 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:27.999034 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:27.999217 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:27.999322 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:27.999434 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:27.999456 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 12:25:27.999475 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:28.001680 19044 raft_consensus.cc:1261] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 1 FOLLOWER]: Refusing update from remote peer f18236bfa0474699b1853b88d7c8e75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:28.001681 18913 raft_consensus.cc:1261] T 9413b9bdea2e4143b3c87b28cfca303c P fd75abc7e27f478ea4be01825992c364 [term 1 FOLLOWER]: Refusing update from remote peer f18236bfa0474699b1853b88d7c8e75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:28.002082 19241 consensus_queue.cc:1035] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "da8d6af780d24b1d801768eaed6a2962" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:28.002254 19245 consensus_queue.cc:1035] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75abc7e27f478ea4be01825992c364" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:28.004958 19248 mvcc.cc:205] Tried to move back new op lower bound from 7333229068292104192 to 7333229068198916096. Current Snapshot: MvccSnapshot[applied={T|T < 7333229068292104192}]
I20260925 12:25:28.004981 19250 mvcc.cc:205] Tried to move back new op lower bound from 7333229068292104192 to 7333229068198916096. Current Snapshot: MvccSnapshot[applied={T|T < 7333229068292104192}]
I20260925 12:25:28.007925 18770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:28.008039 18770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:28.008858 18770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30a26d63386442ffb24d76e106f63842: Sending DeleteTablet for 3 replicas of tablet 9413b9bdea2e4143b3c87b28cfca303c
I20260925 12:25:28.009187 19155 tablet_service.cc:1514] Processing DeleteTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:28 UTC) from {username='slave'} at 127.0.0.1:33576
I20260925 12:25:28.009308 19024 tablet_service.cc:1514] Processing DeleteTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:28 UTC) from {username='slave'} at 127.0.0.1:36012
I20260925 12:25:28.009539 19259 tablet_replica.cc:330] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: stopping tablet replica
I20260925 12:25:28.009670 19259 raft_consensus.cc:2229] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:28.009650 18893 tablet_service.cc:1514] Processing DeleteTablet for tablet 9413b9bdea2e4143b3c87b28cfca303c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:28 UTC) from {username='slave'} at 127.0.0.1:50214
I20260925 12:25:28.009775 19259 raft_consensus.cc:2258] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:28.009918 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18831
I20260925 12:25:28.010231 19259 ts_tablet_manager.cc:1905] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:28.016074 19259 ts_tablet_manager.cc:1918] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:28.016186 19259 log.cc:1196] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/9413b9bdea2e4143b3c87b28cfca303c
I20260925 12:25:28.016389 19258 tablet_replica.cc:330] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: stopping tablet replica
I20260925 12:25:28.016489 19259 ts_tablet_manager.cc:1939] T 9413b9bdea2e4143b3c87b28cfca303c P da8d6af780d24b1d801768eaed6a2962: Deleting consensus metadata
I20260925 12:25:28.016621 19258 raft_consensus.cc:2229] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:25:28.016815 19258 raft_consensus.cc:2258] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:28.016983 18755 catalog_manager.cc:4841] TS da8d6af780d24b1d801768eaed6a2962 (127.1.47.130:44553): tablet 9413b9bdea2e4143b3c87b28cfca303c (table test-table [id=8b527728f450458ba9a170e8717adefe]) successfully deleted
W20260925 12:25:28.017279 18756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:36191 (error 108)
I20260925 12:25:28.017498 19258 ts_tablet_manager.cc:1905] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:28.017849 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:28.017724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36191 (local address 127.0.0.1:50224)
0925 12:25:28.017770 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:28.017813 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 12:25:28.017982 18756 catalog_manager.cc:4584] TS fd75abc7e27f478ea4be01825992c364 (127.1.47.129:36191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9413b9bdea2e4143b3c87b28cfca303c: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36191: connect: Connection refused (error 111)
I20260925 12:25:28.018337 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18962
I20260925 12:25:28.020745 19258 ts_tablet_manager.cc:1918] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:28.020840 19258 log.cc:1196] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/9413b9bdea2e4143b3c87b28cfca303c
I20260925 12:25:28.021087 19258 ts_tablet_manager.cc:1939] T 9413b9bdea2e4143b3c87b28cfca303c P f18236bfa0474699b1853b88d7c8e75b: Deleting consensus metadata
I20260925 12:25:28.021832 18758 catalog_manager.cc:4841] TS f18236bfa0474699b1853b88d7c8e75b (127.1.47.131:38251): tablet 9413b9bdea2e4143b3c87b28cfca303c (table test-table [id=8b527728f450458ba9a170e8717adefe]) successfully deleted
I20260925 12:25:28.024639 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19093
I20260925 12:25:28.030089 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 18740
2026-09-25T12:25:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (700 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:28 dist-test-slave-l9ww krb5kdc[19264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:28 dist-test-slave-l9ww krb5kdc[19264](info): set up 2 sockets
Sep 25 12:25:28 dist-test-slave-l9ww krb5kdc[19264](info): commencing operation
krb5kdc: starting...
W20260925 12:25:29.087674 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:29.210767 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46303
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:46303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:29.291874 19277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:29.292091 19277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:29.292114 19277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:29.293291 19277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:29.293339 19277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:29.293354 19277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:29.293366 19277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:29.293385 19277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:29.294867 19277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:46303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:46303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:29.295128 19277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:29.295384 19277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:29.300202 19285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.300292 19282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.300207 19283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:29.300527 19277 server_base.cc:1034] running on GCE node
I20260925 12:25:29.300771 19277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:29.301082 19277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:29.302227 19277 hybrid_clock.cc:648] HybridClock initialized: now 1790339129302204 us; error 43 us; skew 500 ppm
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:29.304720 19277 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:29.305225 19277 webserver.cc:460] Webserver started at http://127.1.47.190:34689/ using document root <none> and password file <none>
I20260925 12:25:29.305431 19277 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:29.305473 19277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:29.305565 19277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:29.306627 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "299be47cdf6c48dda2ba53da532063b8"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "064b3db78e9840ec8539a78a1973ba53"
server_key_iv: "d4f47cee2b4feac8bb0c1274d1b47827"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.306941 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "299be47cdf6c48dda2ba53da532063b8"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "064b3db78e9840ec8539a78a1973ba53"
server_key_iv: "d4f47cee2b4feac8bb0c1274d1b47827"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.308314 19277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:29.309085 19292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.309266 19277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:29.309340 19277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "299be47cdf6c48dda2ba53da532063b8"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "064b3db78e9840ec8539a78a1973ba53"
server_key_iv: "d4f47cee2b4feac8bb0c1274d1b47827"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.309407 19277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:29.321305 19277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:29.323251 19277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:29.323376 19277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:29.327574 19277 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:46303
I20260925 12:25:29.327643 19344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:46303 every 8 connection(s)
I20260925 12:25:29.328059 19277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:29.329200 19345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:29.331960 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: Bootstrap starting.
I20260925 12:25:29.332576 19345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:29.332911 19345 log.cc:824] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:29.333887 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: No bootstrap required, opened a new log
I20260925 12:25:29.335683 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19277
I20260925 12:25:29.335783 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:29.336001 1214 external_mini_cluster.cc:1428] Setting key 2c61179da4b26ac6af138da033599079
I20260925 12:25:29.336616 19345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } }
I20260925 12:25:29.336815 19345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:29.336861 19345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299be47cdf6c48dda2ba53da532063b8, State: Initialized, Role: FOLLOWER
I20260925 12:25:29.337003 19345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } }
I20260925 12:25:29.337101 19345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:29.337139 19345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:29.337189 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:29.338399 19345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } }
I20260925 12:25:29.338521 19345 leader_election.cc:302] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299be47cdf6c48dda2ba53da532063b8; no voters:
I20260925 12:25:29.338675 19345 leader_election.cc:288] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:29.338702 19350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:29.338804 19350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [term 1 LEADER]: Becoming Leader. State: Replica: 299be47cdf6c48dda2ba53da532063b8, State: Running, Role: LEADER
I20260925 12:25:29.338896 19350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } }
I20260925 12:25:29.338937 19345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:29.339217 19350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299be47cdf6c48dda2ba53da532063b8. Latest consensus state: current_term: 1 leader_uuid: "299be47cdf6c48dda2ba53da532063b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } } }
I20260925 12:25:29.339294 19350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:29.339227 19351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "299be47cdf6c48dda2ba53da532063b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299be47cdf6c48dda2ba53da532063b8" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 46303 } } }
I20260925 12:25:29.339466 19351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8 [sys.catalog]: This master's current role is: LEADER
W20260925 12:25:29.340663 19365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:25:29.340718 19365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:25:29.340772 19355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:29.340931 19355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:29.342531 19355 catalog_manager.cc:1347] Generated new cluster ID: 74a36e2081364de3bcdb998b1317a01f
I20260925 12:25:29.342588 19355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:29.347376 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.336555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.1.47.190:46303)
0925 12:25:29.336751 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.336761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.336799 (+ 38us) server_negotiation.cc:407] Connection header received
0925 12:25:29.336929 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.336937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.337003 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.337152 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:29.338152 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.339425 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.340710 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.341258 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.341385 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.344084 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:29.344088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:29.344089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:29.344107 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:29.345823 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.346223 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.346226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.346226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.346328 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.346548 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.346549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.346550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.346787 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:29.346978 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.347044 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.347186 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260925 12:25:29.367081 19355 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:29.367654 19355 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:29.371526 19355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: Generated new TSK 0
I20260925 12:25:29.371737 19355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:25:29.402412 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:29.502560 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:29.502846 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:29.502910 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:29.504199 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:29.504266 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:29.504313 19371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:29.505918 19371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:29.506191 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:29.506450 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:29.511636 19379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.511632 19377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.511648 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:29.511778 19371 server_base.cc:1034] running on GCE node
I20260925 12:25:29.512087 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:29.512355 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:29.513501 19371 hybrid_clock.cc:648] HybridClock initialized: now 1790339129513485 us; error 32 us; skew 500 ppm
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:29.516163 19371 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:25:29.516647 19371 webserver.cc:460] Webserver started at http://127.1.47.129:44917/ using document root <none> and password file <none>
I20260925 12:25:29.516884 19371 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:29.516932 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:29.517103 19371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:29.518304 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "209d465bc9bc4c5db637d0a093d3ddda"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "5bead0935d55c1bdaf82474339c6906c"
server_key_iv: "7ddd248a6503f95fa7907b6a99e54659"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.518674 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "209d465bc9bc4c5db637d0a093d3ddda"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "5bead0935d55c1bdaf82474339c6906c"
server_key_iv: "7ddd248a6503f95fa7907b6a99e54659"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.520059 19371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 12:25:29.520843 19386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.521080 19371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:29.521173 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "209d465bc9bc4c5db637d0a093d3ddda"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "5bead0935d55c1bdaf82474339c6906c"
server_key_iv: "7ddd248a6503f95fa7907b6a99e54659"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.521283 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:29.547473 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:29.549744 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:29.549911 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:29.550199 19371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:29.550582 19371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:29.550643 19371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.550705 19371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:29.550738 19371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.557163 19371 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43523
I20260925 12:25:29.557231 19499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43523 every 8 connection(s)
I20260925 12:25:29.557586 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:29.558531 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19371
I20260925 12:25:29.558637 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:29.558893 1214 external_mini_cluster.cc:1428] Setting key 71c0fab9777feb9785a86d6913ecba46
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
I20260925 12:25:29.567975 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.558426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:38883 (local address 127.1.47.190:46303)
0925 12:25:29.558480 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.558482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.558766 (+ 284us) server_negotiation.cc:407] Connection header received
0925 12:25:29.559470 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.559471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.559483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.559540 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:29.560664 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.561183 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.562252 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.562464 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.562556 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.564837 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:29.564838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:29.564839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:29.564847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:29.566339 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.566763 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.566764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.566765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.566831 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.567113 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.567114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.567114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.567272 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:29.567339 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.567755 (+ 416us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.567850 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:25:29.568306 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.558409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46303 (local address 127.1.47.129:38883)
0925 12:25:29.558625 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.558678 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.559326 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.559621 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.559626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.560319 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:29.560535 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.560540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.561295 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.561296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.562149 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.562152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.562570 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.562571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.562715 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.563296 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:29.563305 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:29.564723 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:29.566434 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.566442 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.566444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.566669 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.566920 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.566921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.566922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.567032 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.567364 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:29.567379 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:29.567602 (+ 223us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.567731 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.567920 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260925 12:25:29.568909 19500 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46303
I20260925 12:25:29.569052 19500 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:29.569264 19500 heartbeater.cc:511] Master 127.1.47.190:46303 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
I20260925 12:25:29.569900 19309 ts_manager.cc:194] Registered new tserver with Master: 209d465bc9bc4c5db637d0a093d3ddda (127.1.47.129:43523)
I20260925 12:25:29.570540 19309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:38883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:25:29.584476 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:29.667572 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:29.667878 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:29.667905 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:29.669163 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:29.669206 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:29.669234 19505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:29.670689 19505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:29.670902 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:29.671082 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:29.676270 19511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.676290 19510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:29.676311 19505 server_base.cc:1034] running on GCE node
W20260925 12:25:29.676273 19513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:29.676697 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:29.676963 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:29.678134 19505 hybrid_clock.cc:648] HybridClock initialized: now 1790339129678131 us; error 38 us; skew 500 ppm
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:29.680713 19505 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:25:29.681154 19505 webserver.cc:460] Webserver started at http://127.1.47.130:37861/ using document root <none> and password file <none>
I20260925 12:25:29.681392 19505 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:29.681461 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:29.681609 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:29.682649 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "d2b233a085b9440ea44ea216d47c621a"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "297e3ca0dd00cd872810d9e917756b50"
server_key_iv: "21df28a41c70a0495d9a792c524c7265"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.683015 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "d2b233a085b9440ea44ea216d47c621a"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "297e3ca0dd00cd872810d9e917756b50"
server_key_iv: "21df28a41c70a0495d9a792c524c7265"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.684386 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:29.685081 19520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.685256 19505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:29.685360 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "d2b233a085b9440ea44ea216d47c621a"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "297e3ca0dd00cd872810d9e917756b50"
server_key_iv: "21df28a41c70a0495d9a792c524c7265"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.685432 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:29.712464 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:29.714859 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:29.715037 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:29.715281 19505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:29.715579 19505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:29.715632 19505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.715678 19505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:29.715704 19505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.721640 19505 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:43675
I20260925 12:25:29.721692 19633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:43675 every 8 connection(s)
I20260925 12:25:29.722002 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:29.729926 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19505
I20260925 12:25:29.730054 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:29.730279 1214 external_mini_cluster.cc:1428] Setting key 0354168af72ae7ad023af3c33d5f417a
I20260925 12:25:29.731313 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.722660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:38603 (local address 127.1.47.190:46303)
0925 12:25:29.722704 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.722706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.722913 (+ 207us) server_negotiation.cc:407] Connection header received
0925 12:25:29.723401 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.723402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.723411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.723475 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:29.724580 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.725070 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.726107 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.726292 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.726346 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.728496 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:29.728497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:29.728498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:29.728505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:29.729669 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.730062 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.730064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.730064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.730151 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.730460 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.730461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.730462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.730633 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:29.730714 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.731062 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.731185 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:25:29.731407 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.722618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46303 (local address 127.1.47.130:38603)
0925 12:25:29.722779 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.722822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.723311 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.723555 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.723558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.724184 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:29.724498 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.724504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.725157 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.725158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.726037 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.726039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.726373 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.726374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.726473 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.726957 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:29.726964 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:29.728409 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:29.729738 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.729744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.729746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.729971 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.730228 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.730229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.730229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.730378 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.730717 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:29.730724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:29.730885 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.731003 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.731183 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260925 12:25:29.732008 19634 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46303
I20260925 12:25:29.732121 19634 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:29.732295 19634 heartbeater.cc:511] Master 127.1.47.190:46303 requested a full tablet report, sending...
I20260925 12:25:29.732733 19309 ts_manager.cc:194] Registered new tserver with Master: d2b233a085b9440ea44ea216d47c621a (127.1.47.130:43675)
I20260925 12:25:29.733274 19309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:38603
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:25:29.755108 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:29.838291 19639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:29.838550 19639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:29.838590 19639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:29.840091 19639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:29.840154 19639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:29.840201 19639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:29.841779 19639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:29.842061 19639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:29.842311 19639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:29.847441 19645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.847451 19644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:29.847468 19647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:29.847990 19639 server_base.cc:1034] running on GCE node
I20260925 12:25:29.848220 19639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:29.848500 19639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:29.849653 19639 hybrid_clock.cc:648] HybridClock initialized: now 1790339129849627 us; error 36 us; skew 500 ppm
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:29.852404 19639 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:25:29.852851 19639 webserver.cc:460] Webserver started at http://127.1.47.131:44867/ using document root <none> and password file <none>
I20260925 12:25:29.853114 19639 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:29.853183 19639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:29.853338 19639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:29.854405 19639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "95ff709f698240b39c36fbfbce5510b1"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "b147c880f10fe007174ca158431b570f"
server_key_iv: "0ed14dfe295d3876420f81da9ecda78d"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.854772 19639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "95ff709f698240b39c36fbfbce5510b1"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "b147c880f10fe007174ca158431b570f"
server_key_iv: "0ed14dfe295d3876420f81da9ecda78d"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.856169 19639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 12:25:29.856982 19654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.857151 19639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:29.857259 19639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "95ff709f698240b39c36fbfbce5510b1"
format_stamp: "Formatted at 2026-09-25 12:25:29 on dist-test-slave-l9ww"
server_key: "b147c880f10fe007174ca158431b570f"
server_key_iv: "0ed14dfe295d3876420f81da9ecda78d"
server_key_version: "encryptionkey@0"
I20260925 12:25:29.857334 19639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:29.872217 19639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:29.874739 19639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:29.874931 19639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:29.875187 19639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:29.875510 19639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:29.875566 19639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.875607 19639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:29.875640 19639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:29.881726 19639 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:36311
I20260925 12:25:29.881848 19767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:36311 every 8 connection(s)
I20260925 12:25:29.882164 19639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
Sep 25 12:25:29 dist-test-slave-l9ww krb5kdc[19264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:29.890668 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19639
I20260925 12:25:29.890794 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:29.890992 1214 external_mini_cluster.cc:1428] Setting key 9b6de2aadb25ca2d3d668b7269317d25
I20260925 12:25:29.893144 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.883009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:58103 (local address 127.1.47.190:46303)
0925 12:25:29.883132 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.883167 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.883332 (+ 165us) server_negotiation.cc:407] Connection header received
0925 12:25:29.883986 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.883987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.884002 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.884070 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:29.885221 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.885799 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.886967 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.887214 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.887302 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.889811 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:29.889812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:29.889813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:29.889820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:29.891570 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.892198 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.892198 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.892199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.892288 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.892547 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.892547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.892548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.892654 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:29.892705 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.892998 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.893063 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 12:25:29.893352 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.882966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:46303 (local address 127.1.47.131:58103)
0925 12:25:29.883164 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.883226 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.883799 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.884156 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.884165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.884836 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:29.885057 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.885063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.885908 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.885909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.886850 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.886852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.887324 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.887325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.887464 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.888087 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:29.888096 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:29.889711 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:29.891644 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.891669 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.891672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.892092 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.892371 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:29.892377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:29.892377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:29.892492 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:29.892730 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:29.892736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:29.892866 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.892954 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.893085 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260925 12:25:29.893783 19768 heartbeater.cc:347] Connected to a master server at 127.1.47.190:46303
I20260925 12:25:29.893893 19768 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:29.894083 19768 heartbeater.cc:511] Master 127.1.47.190:46303 requested a full tablet report, sending...
I20260925 12:25:29.894462 19309 ts_manager.cc:194] Registered new tserver with Master: 95ff709f698240b39c36fbfbce5510b1 (127.1.47.131:36311)
I20260925 12:25:29.894850 19309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:58103
I20260925 12:25:29.902926 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:29.910719 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.904375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.1.47.190:46303)
0925 12:25:29.904464 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.904467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.904529 (+ 62us) server_negotiation.cc:407] Connection header received
0925 12:25:29.904609 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.904609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.904619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.904665 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:29.905247 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.905740 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.906558 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.906771 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.906858 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.908303 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:29.908304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:29.908305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:29.908311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:29.909572 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.909954 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.909955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.909955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.910009 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:29.910192 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:29.910193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:29.910193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:29.910324 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:29.910412 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.910540 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.910591 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 12:25:29.912355 19309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:29.913935 19309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:29.920861 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.1.47.131:36311)
0925 12:25:29.917031 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.917035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.917043 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:25:29.917082 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.917084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.917099 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.917296 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:29.917377 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918475 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.919660 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920373 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.920448 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.920506 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.920728 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.920772 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260925 12:25:29.920863 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:36311 (local address 127.0.0.1:57448)
0925 12:25:29.916655 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.916663 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.916699 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.917188 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.917188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.917190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:29.917227 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.917228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918610 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.918611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.919573 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.919575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920472 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.920473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.920560 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.920576 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.920685 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.920766 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":20,"thread_start_us":114,"threads_started":1}
I20260925 12:25:29.921082 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:43675 (local address 127.0.0.1:56148)
0925 12:25:29.916770 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.916778 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.916812 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.917246 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.917247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.917249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:29.917326 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.917328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918359 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.918361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.919772 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.919775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920801 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.920802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.920886 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.920890 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.920945 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.920991 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":90,"threads_started":1}
I20260925 12:25:29.921509 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43523 (local address 127.0.0.1:40430)
0925 12:25:29.916450 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.916491 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.916554 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.917139 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.917142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.917144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:29.917217 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.917219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918588 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.918590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.919919 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.919921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920787 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.920788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.920872 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.920878 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.920916 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.920963 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":29,"thread_start_us":45,"threads_started":1}
I20260925 12:25:29.921624 19702 tablet_service.cc:1467] Processing CreateTablet for tablet db08532f1ac5407289b9ec3a7332e9ab (DEFAULT_TABLE table=test-table [id=0e2c33291b214678beb07392abab1022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:29.921929 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.1.47.129:43523)
0925 12:25:29.916944 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.916949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.916959 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:29.917006 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.917008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.917024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.917294 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:29.917370 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918488 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.919996 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920673 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.921679 (+ 1006us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.921753 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.921779 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.921850 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":92,"threads_started":1}
I20260925 12:25:29.922034 19702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db08532f1ac5407289b9ec3a7332e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:29.922576 19434 tablet_service.cc:1467] Processing CreateTablet for tablet db08532f1ac5407289b9ec3a7332e9ab (DEFAULT_TABLE table=test-table [id=0e2c33291b214678beb07392abab1022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:29.922932 19434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db08532f1ac5407289b9ec3a7332e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:29.924093 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.916665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.1.47.130:43675)
0925 12:25:29.916848 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.916852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.916863 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:25:29.916902 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.916904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.916995 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.917095 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:29.917442 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.918224 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.919915 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.920606 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.920716 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.920772 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.923941 (+ 3169us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.923999 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260925 12:25:29.924772 19568 tablet_service.cc:1467] Processing CreateTablet for tablet db08532f1ac5407289b9ec3a7332e9ab (DEFAULT_TABLE table=test-table [id=0e2c33291b214678beb07392abab1022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:29.925176 19568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db08532f1ac5407289b9ec3a7332e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:29.926608 19786 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: Bootstrap starting.
I20260925 12:25:29.926743 19785 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Bootstrap starting.
I20260925 12:25:29.927206 19786 tablet_bootstrap.cc:654] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:29.927476 19785 tablet_bootstrap.cc:654] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:29.927784 19786 log.cc:824] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:29.928162 19785 log.cc:824] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:29.928566 19786 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: No bootstrap required, opened a new log
I20260925 12:25:29.928656 19786 ts_tablet_manager.cc:1397] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:25:29.929008 19785 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: No bootstrap required, opened a new log
I20260925 12:25:29.929090 19785 ts_tablet_manager.cc:1397] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:25:29.929208 19789 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Bootstrap starting.
I20260925 12:25:29.929869 19789 tablet_bootstrap.cc:654] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:29.930440 19789 log.cc:824] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:29.930788 19786 raft_consensus.cc:348] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.930961 19786 raft_consensus.cc:374] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:29.931010 19786 raft_consensus.cc:729] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209d465bc9bc4c5db637d0a093d3ddda, State: Initialized, Role: FOLLOWER
I20260925 12:25:29.931025 19785 raft_consensus.cc:348] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.931141 19786 consensus_queue.cc:260] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.931229 19785 raft_consensus.cc:374] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:29.931311 19785 raft_consensus.cc:729] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ff709f698240b39c36fbfbce5510b1, State: Initialized, Role: FOLLOWER
I20260925 12:25:29.931443 19786 ts_tablet_manager.cc:1428] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:25:29.931452 19785 consensus_queue.cc:260] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.931507 19500 heartbeater.cc:503] Master 127.1.47.190:46303 was elected leader, sending a full tablet report...
I20260925 12:25:29.931524 19789 tablet_bootstrap.cc:492] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: No bootstrap required, opened a new log
I20260925 12:25:29.931629 19789 ts_tablet_manager.cc:1397] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:25:29.931788 19785 ts_tablet_manager.cc:1428] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:25:29.932253 19768 heartbeater.cc:503] Master 127.1.47.190:46303 was elected leader, sending a full tablet report...
I20260925 12:25:29.933921 19789 raft_consensus.cc:348] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.934054 19789 raft_consensus.cc:374] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:29.934077 19789 raft_consensus.cc:729] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b233a085b9440ea44ea216d47c621a, State: Initialized, Role: FOLLOWER
I20260925 12:25:29.934190 19789 consensus_queue.cc:260] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.934448 19789 ts_tablet_manager.cc:1428] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:25:29.934477 19634 heartbeater.cc:503] Master 127.1.47.190:46303 was elected leader, sending a full tablet report...
W20260925 12:25:29.972572 19635 tablet.cc:2369] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:29.984313 19792 raft_consensus.cc:482] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:29.984448 19792 raft_consensus.cc:504] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.984936 19792 leader_election.cc:288] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 209d465bc9bc4c5db637d0a093d3ddda (127.1.47.129:43523), d2b233a085b9440ea44ea216d47c621a (127.1.47.130:43675)
I20260925 12:25:29.988194 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.985116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43523 (local address 127.1.47.131:41843)
0925 12:25:29.985191 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.985205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.985285 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.985479 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.985481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.985483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:29.985564 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.985566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.986339 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.986340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.987147 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.987149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.987903 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.987904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.987998 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.988002 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.988074 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.988121 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 12:25:29.988238 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.985249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:41843 (local address 127.1.47.129:43523)
0925 12:25:29.985301 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.985303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.985316 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:25:29.985353 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.985354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.985368 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.985445 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:29.985645 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.986244 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.987253 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.987791 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.987848 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.987905 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.988113 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.988160 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:25:29.988397 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.985194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:43675 (local address 127.1.47.131:57465)
0925 12:25:29.985394 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:29.985406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:29.985467 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:29.985640 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:29.985641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:29.985643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:29.985720 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.985721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.986464 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.986465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.987387 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:29.987390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.988116 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.988118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:29.988195 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.988200 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:29.988268 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 12:25:29.988309 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":118,"threads_started":1}
I20260925 12:25:29.988695 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:29.985274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:57465 (local address 127.1.47.130:43675)
0925 12:25:29.985326 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:29.985329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:29.985486 (+ 157us) server_negotiation.cc:407] Connection header received
0925 12:25:29.985519 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:29.985520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:29.985535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:29.985597 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:29.985836 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.986357 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.987529 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:29.988013 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:29.988084 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:29.988105 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:29.988579 (+ 474us) server_negotiation.cc:299] Negotiation successful
0925 12:25:29.988626 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 12:25:29.988654 19454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db08532f1ac5407289b9ec3a7332e9ab" candidate_uuid: "95ff709f698240b39c36fbfbce5510b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209d465bc9bc4c5db637d0a093d3ddda" is_pre_election: true
I20260925 12:25:29.988817 19454 raft_consensus.cc:2454] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ff709f698240b39c36fbfbce5510b1 in term 0.
I20260925 12:25:29.988991 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db08532f1ac5407289b9ec3a7332e9ab" candidate_uuid: "95ff709f698240b39c36fbfbce5510b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b233a085b9440ea44ea216d47c621a" is_pre_election: true
I20260925 12:25:29.989077 19658 leader_election.cc:302] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209d465bc9bc4c5db637d0a093d3ddda, 95ff709f698240b39c36fbfbce5510b1; no voters:
I20260925 12:25:29.989179 19588 raft_consensus.cc:2454] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ff709f698240b39c36fbfbce5510b1 in term 0.
I20260925 12:25:29.989238 19792 raft_consensus.cc:2781] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:29.989298 19792 raft_consensus.cc:482] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:29.989327 19792 raft_consensus.cc:3037] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:29.990114 19792 raft_consensus.cc:504] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.990268 19792 leader_election.cc:288] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [CANDIDATE]: Term 1 election: Requested vote from peers 209d465bc9bc4c5db637d0a093d3ddda (127.1.47.129:43523), d2b233a085b9440ea44ea216d47c621a (127.1.47.130:43675)
I20260925 12:25:29.990418 19454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db08532f1ac5407289b9ec3a7332e9ab" candidate_uuid: "95ff709f698240b39c36fbfbce5510b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209d465bc9bc4c5db637d0a093d3ddda"
I20260925 12:25:29.990520 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db08532f1ac5407289b9ec3a7332e9ab" candidate_uuid: "95ff709f698240b39c36fbfbce5510b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b233a085b9440ea44ea216d47c621a"
I20260925 12:25:29.990513 19454 raft_consensus.cc:3037] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:29.990602 19588 raft_consensus.cc:3037] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:29.991271 19454 raft_consensus.cc:2454] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ff709f698240b39c36fbfbce5510b1 in term 1.
I20260925 12:25:29.991288 19588 raft_consensus.cc:2454] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ff709f698240b39c36fbfbce5510b1 in term 1.
I20260925 12:25:29.991446 19658 leader_election.cc:302] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209d465bc9bc4c5db637d0a093d3ddda, 95ff709f698240b39c36fbfbce5510b1; no voters:
I20260925 12:25:29.991580 19792 raft_consensus.cc:2781] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:29.991760 19792 raft_consensus.cc:686] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 1 LEADER]: Becoming Leader. State: Replica: 95ff709f698240b39c36fbfbce5510b1, State: Running, Role: LEADER
I20260925 12:25:29.991863 19792 consensus_queue.cc:237] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } }
I20260925 12:25:29.992681 19309 catalog_manager.cc:5495] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95ff709f698240b39c36fbfbce5510b1 (127.1.47.131). New cstate: current_term: 1 leader_uuid: "95ff709f698240b39c36fbfbce5510b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ff709f698240b39c36fbfbce5510b1" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 36311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 } health_report { overall_health: UNKNOWN } } }
I20260925 12:25:30.012493 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.010138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.1.47.131:36311)
0925 12:25:30.010204 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.010206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.010214 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:25:30.010247 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.010248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.010266 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.010330 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:30.010449 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.010924 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.011667 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.011933 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.011994 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.012153 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:30.012254 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:30.012331 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.012413 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.012422 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 12:25:30.014357 19454 raft_consensus.cc:1261] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 1 FOLLOWER]: Refusing update from remote peer 95ff709f698240b39c36fbfbce5510b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:30.014356 19588 raft_consensus.cc:1261] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 1 FOLLOWER]: Refusing update from remote peer 95ff709f698240b39c36fbfbce5510b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:30.014731 19792 consensus_queue.cc:1035] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2b233a085b9440ea44ea216d47c621a" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:30.014861 19795 consensus_queue.cc:1035] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "209d465bc9bc4c5db637d0a093d3ddda" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:30.017567 19798 mvcc.cc:205] Tried to move back new op lower bound from 7333229076536025088 to 7333229076447457280. Current Snapshot: MvccSnapshot[applied={T|T < 7333229076536025088}]
I20260925 12:25:30.017750 19799 mvcc.cc:205] Tried to move back new op lower bound from 7333229076536025088 to 7333229076447457280. Current Snapshot: MvccSnapshot[applied={T|T < 7333229076536025088}]
I20260925 12:25:30.020812 19307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:30.020934 19307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:30.021711 19307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299be47cdf6c48dda2ba53da532063b8: Sending DeleteTablet for 3 replicas of tablet db08532f1ac5407289b9ec3a7332e9ab
I20260925 12:25:30.022063 19434 tablet_service.cc:1514] Processing DeleteTablet for tablet db08532f1ac5407289b9ec3a7332e9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40430
I20260925 12:25:30.022197 19702 tablet_service.cc:1514] Processing DeleteTablet for tablet db08532f1ac5407289b9ec3a7332e9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:57448
I20260925 12:25:30.022200 19568 tablet_service.cc:1514] Processing DeleteTablet for tablet db08532f1ac5407289b9ec3a7332e9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:56148
I20260925 12:25:30.022472 19808 tablet_replica.cc:330] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda: stopping tablet replica
I20260925 12:25:30.022596 19808 raft_consensus.cc:2229] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:30.022626 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19371
I20260925 12:25:30.022716 19808 raft_consensus.cc:2258] T db08532f1ac5407289b9ec3a7332e9ab P 209d465bc9bc4c5db637d0a093d3ddda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:30.028038 19810 tablet_replica.cc:330] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: stopping tablet replica
I20260925 12:25:30.028038 19809 tablet_replica.cc:330] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: stopping tablet replica
I20260925 12:25:30.028165 19810 raft_consensus.cc:2229] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:30.028164 19809 raft_consensus.cc:2229] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 12:25:30.028283 19810 raft_consensus.cc:2258] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:30.028297 19809 raft_consensus.cc:2258] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:30.028801 19809 ts_tablet_manager.cc:1905] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:30.028801 19810 ts_tablet_manager.cc:1905] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 12:25:30.029963 19296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43523 (error 108)
I20260925 12:25:30.030589 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19505
I20260925 12:25:30.030601 19810 ts_tablet_manager.cc:1918] T db08532f1ac5407289b9ec3a7332e9ab P d2b233a085b9440ea44ea216d47c621a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:30.030599 19809 ts_tablet_manager.cc:1918] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:30.031976 19809 log.cc:1196] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/db08532f1ac5407289b9ec3a7332e9ab
I20260925 12:25:30.032336 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.032102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43523 (local address 127.0.0.1:40446)
0925 12:25:30.032239 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.032293 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260925 12:25:30.032336 19809 ts_tablet_manager.cc:1939] T db08532f1ac5407289b9ec3a7332e9ab P 95ff709f698240b39c36fbfbce5510b1: Deleting consensus metadata
I20260925 12:25:30.032837 19294 catalog_manager.cc:4841] TS 95ff709f698240b39c36fbfbce5510b1 (127.1.47.131:36311): tablet db08532f1ac5407289b9ec3a7332e9ab (table test-table [id=0e2c33291b214678beb07392abab1022]) successfully deleted
W20260925 12:25:30.033304 19296 catalog_manager.cc:4584] TS 209d465bc9bc4c5db637d0a093d3ddda (127.1.47.129:43523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db08532f1ac5407289b9ec3a7332e9ab: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43523: connect: Connection refused (error 111)
I20260925 12:25:30.036842 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.036695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:43675 (local address 127.0.0.1:56162)
0925 12:25:30.036763 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.036795 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.130:43675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 12:25:30.036934 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19639
I20260925 12:25:30.042116 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19277
2026-09-25T12:25:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2012 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T12:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:30Z Disabled control of system clock
I20260925 12:25:30.069931 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37983
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:30.150600 19815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:30.150823 19815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:30.150853 19815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:30.152032 19815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:30.152070 19815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:30.152082 19815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:30.152096 19815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:30.152107 19815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:30.153376 19815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:30.153623 19815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:30.153906 19815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:30.158610 19820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.158627 19821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.158622 19823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:30.158860 19815 server_base.cc:1034] running on GCE node
I20260925 12:25:30.159109 19815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:30.159358 19815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:30.160501 19815 hybrid_clock.cc:648] HybridClock initialized: now 1790339130160486 us; error 30 us; skew 500 ppm
I20260925 12:25:30.161688 19815 webserver.cc:460] Webserver started at http://127.1.47.190:42227/ using document root <none> and password file <none>
I20260925 12:25:30.161962 19815 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:30.162022 19815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:30.162125 19815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:30.163221 19815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "b0dbc0bde1fd4b9382f99346402a6146"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "7dbb8fc948cefdc5597cbabe91d2e91a"
server_key_iv: "90cc9f9f04828c522c29a373e0a51ff0"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.163522 19815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "b0dbc0bde1fd4b9382f99346402a6146"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "7dbb8fc948cefdc5597cbabe91d2e91a"
server_key_iv: "90cc9f9f04828c522c29a373e0a51ff0"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.164847 19815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:30.165582 19829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.165776 19815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:30.165848 19815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "b0dbc0bde1fd4b9382f99346402a6146"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "7dbb8fc948cefdc5597cbabe91d2e91a"
server_key_iv: "90cc9f9f04828c522c29a373e0a51ff0"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.165911 19815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:30.178445 19815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:30.178735 19815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:30.178844 19815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:30.182711 19815 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37983
I20260925 12:25:30.182746 19881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37983 every 8 connection(s)
I20260925 12:25:30.183133 19815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:30.184188 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19815
I20260925 12:25:30.184293 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:30.184296 19882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:30.184518 1214 external_mini_cluster.cc:1428] Setting key 5791a5e362e4d7ef73569094bbf8c330
I20260925 12:25:30.187188 19882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: Bootstrap starting.
I20260925 12:25:30.187775 19882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:30.188126 19882 log.cc:824] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:30.188828 19882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: No bootstrap required, opened a new log
I20260925 12:25:30.189970 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.185077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.1.47.190:37983)
0925 12:25:30.185311 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.185319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.185343 (+ 24us) server_negotiation.cc:407] Connection header received
0925 12:25:30.185793 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.185799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.185866 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.186022 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:30.186251 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.187114 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.187989 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.188453 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.188592 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.189322 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:30.189326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:30.189327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:30.189508 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:30.189542 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:30.189553 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:30.189633 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.189712 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.189798 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260925 12:25:30.191025 19882 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } }
I20260925 12:25:30.191166 19882 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:30.191188 19882 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0dbc0bde1fd4b9382f99346402a6146, State: Initialized, Role: FOLLOWER
I20260925 12:25:30.191335 19882 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } }
I20260925 12:25:30.191401 19882 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:30.191443 19882 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:30.191491 19882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:30.192169 19882 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } }
I20260925 12:25:30.192283 19882 leader_election.cc:302] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0dbc0bde1fd4b9382f99346402a6146; no voters:
I20260925 12:25:30.192498 19882 leader_election.cc:288] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:30.192572 19887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:30.192718 19887 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [term 1 LEADER]: Becoming Leader. State: Replica: b0dbc0bde1fd4b9382f99346402a6146, State: Running, Role: LEADER
I20260925 12:25:30.192800 19882 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:30.192809 19887 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } }
I20260925 12:25:30.193274 19889 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0dbc0bde1fd4b9382f99346402a6146. Latest consensus state: current_term: 1 leader_uuid: "b0dbc0bde1fd4b9382f99346402a6146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } } }
I20260925 12:25:30.193275 19888 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0dbc0bde1fd4b9382f99346402a6146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbc0bde1fd4b9382f99346402a6146" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37983 } } }
I20260925 12:25:30.193475 19889 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:30.193485 19888 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:30.193866 19896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:30.194363 19896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:30.195996 19896 catalog_manager.cc:1347] Generated new cluster ID: 107b505b84ec4966b5dcf9a3bfab6b36
I20260925 12:25:30.196048 19896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:30.208885 19896 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:30.209633 19896 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:30.222975 19896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: Generated new TSK 0
I20260925 12:25:30.223246 19896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:30.236338 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--builtin_ntp_servers=127.1.47.148:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:30.317530 19906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:30.317730 19906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:30.317754 19906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:30.318907 19906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:30.318945 19906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:30.318971 19906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:30.320358 19906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:30.320606 19906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:30.320941 19906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:30.325949 19911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.325996 19912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.326052 19914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:30.326154 19906 server_base.cc:1034] running on GCE node
I20260925 12:25:30.326360 19906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:30.326606 19906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:30.327767 19906 hybrid_clock.cc:648] HybridClock initialized: now 1790339130327756 us; error 45 us; skew 500 ppm
I20260925 12:25:30.328992 19906 webserver.cc:460] Webserver started at http://127.1.47.129:40737/ using document root <none> and password file <none>
I20260925 12:25:30.329249 19906 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:30.329321 19906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:30.329481 19906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:30.330564 19906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "3d33674aaa2cbca913b970969239005d"
server_key_iv: "6b97c5e870b709573c1e4e25eb442f19"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.330940 19906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "3d33674aaa2cbca913b970969239005d"
server_key_iv: "6b97c5e870b709573c1e4e25eb442f19"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.332360 19906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:30.333143 19920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.333313 19906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:30.333417 19906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "3d33674aaa2cbca913b970969239005d"
server_key_iv: "6b97c5e870b709573c1e4e25eb442f19"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.333499 19906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:30.348315 19906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:30.348657 19906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:30.348824 19906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:30.349066 19906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:30.349407 19906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:30.349463 19906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.349503 19906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:30.349530 19906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.355480 19906 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43705
I20260925 12:25:30.355526 20033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43705 every 8 connection(s)
I20260925 12:25:30.355929 19906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:30.361119 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.356710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:42725 (local address 127.1.47.190:37983)
0925 12:25:30.356783 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.356785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.357021 (+ 236us) server_negotiation.cc:407] Connection header received
0925 12:25:30.357514 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.357515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.357538 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.357597 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:30.358283 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.358811 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.359766 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.360055 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.360118 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.360895 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:30.360896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:30.360896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:30.360900 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:30.360915 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:30.360920 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:30.360967 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.361029 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.361058 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 12:25:30.361325 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.356712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37983 (local address 127.1.47.129:42725)
0925 12:25:30.356894 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.356929 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.357418 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.357674 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.357678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.357686 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:30.358185 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.358189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.358895 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.358896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.359674 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.359676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.360133 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.360133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.360247 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.360752 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:30.360753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:30.360786 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:30.360809 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:30.360967 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:30.360969 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:30.360971 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.361017 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.361092 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260925 12:25:30.361641 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 19906
I20260925 12:25:30.361752 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:30.361912 20034 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37983
I20260925 12:25:30.362007 20034 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:30.362006 1214 external_mini_cluster.cc:1428] Setting key 17194d608006968339935abcb8132a77
I20260925 12:25:30.362211 20034 heartbeater.cc:511] Master 127.1.47.190:37983 requested a full tablet report, sending...
I20260925 12:25:30.362789 19845 ts_manager.cc:194] Registered new tserver with Master: 0a9fa2be4f064abd88e4dce1fc1f43ea (127.1.47.129:43705)
I20260925 12:25:30.362984 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--builtin_ntp_servers=127.1.47.148:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 12:25:30.363556 19845 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.129:42725
W20260925 12:25:30.452298 20037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:30.452512 20037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:30.452535 20037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:30.453739 20037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:30.453788 20037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:30.453814 20037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:30.455241 20037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:30.455505 20037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:30.455813 20037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:30.461133 20045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:30.461220 20037 server_base.cc:1034] running on GCE node
W20260925 12:25:30.461122 20043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.461122 20042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:30.461578 20037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:30.461891 20037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:30.463091 20037 hybrid_clock.cc:648] HybridClock initialized: now 1790339130463072 us; error 37 us; skew 500 ppm
I20260925 12:25:30.464439 20037 webserver.cc:460] Webserver started at http://127.1.47.130:43063/ using document root <none> and password file <none>
I20260925 12:25:30.464689 20037 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:30.464758 20037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:30.464912 20037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:30.466010 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "1c341ecad76341c09e0bde092d56a7c4"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "8517af83afc5977371475bd24ea033d7"
server_key_iv: "e73d2d046548276f8f0a10084fbac90d"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.466382 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "1c341ecad76341c09e0bde092d56a7c4"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "8517af83afc5977371475bd24ea033d7"
server_key_iv: "e73d2d046548276f8f0a10084fbac90d"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.467681 20037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 12:25:30.468531 20051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.468715 20037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:25:30.468818 20037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "1c341ecad76341c09e0bde092d56a7c4"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "8517af83afc5977371475bd24ea033d7"
server_key_iv: "e73d2d046548276f8f0a10084fbac90d"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.468915 20037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:30.481144 20037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:30.481485 20037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:30.481639 20037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:30.481875 20037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:30.482229 20037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:30.482285 20037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.482328 20037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:30.482355 20037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.488153 20037 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:34063
I20260925 12:25:30.488234 20164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:34063 every 8 connection(s)
I20260925 12:25:30.488603 20037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:25:30.494189 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.489446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:58727 (local address 127.1.47.190:37983)
0925 12:25:30.489524 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.489526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.489807 (+ 281us) server_negotiation.cc:407] Connection header received
0925 12:25:30.490351 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.490352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.490362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.490419 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:30.491146 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.491639 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.492668 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.492850 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.492940 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.493823 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:30.493824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:30.493824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:30.493829 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:30.493845 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:30.493849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:30.493894 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.494104 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.494126 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 12:25:30.494483 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.489426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37983 (local address 127.1.47.130:58727)
0925 12:25:30.489653 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.489710 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.490255 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.490486 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.490490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.490499 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:30.491064 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.491069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.491746 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.491747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.492586 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.492588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.492957 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.492958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.493101 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.493693 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:30.493694 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:30.493723 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:30.493747 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:30.493934 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:30.493940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:30.493991 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.494078 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.494225 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260925 12:25:30.494985 20165 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37983
I20260925 12:25:30.495090 20165 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:30.495278 20165 heartbeater.cc:511] Master 127.1.47.190:37983 requested a full tablet report, sending...
I20260925 12:25:30.495658 19845 ts_manager.cc:194] Registered new tserver with Master: 1c341ecad76341c09e0bde092d56a7c4 (127.1.47.130:34063)
I20260925 12:25:30.496106 19845 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.130:58727
I20260925 12:25:30.498202 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20037
I20260925 12:25:30.498323 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:30.498533 1214 external_mini_cluster.cc:1428] Setting key af3d85a985efbd595b6d71f8648a19fd
I20260925 12:25:30.499689 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--builtin_ntp_servers=127.1.47.148:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:30.581768 20168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:30.581991 20168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:30.582013 20168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:30.583123 20168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:30.583158 20168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:30.583184 20168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:30.584483 20168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:37983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:30.584678 20168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:30.584935 20168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:30.589982 20174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.589982 20176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:30.590014 20173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:30.590226 20168 server_base.cc:1034] running on GCE node
I20260925 12:25:30.590433 20168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:30.590637 20168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:30.591799 20168 hybrid_clock.cc:648] HybridClock initialized: now 1790339130591785 us; error 27 us; skew 500 ppm
I20260925 12:25:30.593318 20168 webserver.cc:460] Webserver started at http://127.1.47.131:34435/ using document root <none> and password file <none>
I20260925 12:25:30.593593 20168 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:30.593640 20168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:30.593824 20168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:30.594975 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "708104f012e1499999ba6ae201bfe593"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "9b0d49f34d7899e0c7d860d21e8e18c9"
server_key_iv: "1a9662b0d5f07c1cfec4d8893454b14e"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.595341 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "708104f012e1499999ba6ae201bfe593"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "9b0d49f34d7899e0c7d860d21e8e18c9"
server_key_iv: "1a9662b0d5f07c1cfec4d8893454b14e"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.596742 20168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:30.597460 20182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.597628 20168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:30.597733 20168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "708104f012e1499999ba6ae201bfe593"
format_stamp: "Formatted at 2026-09-25 12:25:30 on dist-test-slave-l9ww"
server_key: "9b0d49f34d7899e0c7d860d21e8e18c9"
server_key_iv: "1a9662b0d5f07c1cfec4d8893454b14e"
server_key_version: "encryptionkey@0"
I20260925 12:25:30.597806 20168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:30.613536 20168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:30.613849 20168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:30.613970 20168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:30.614158 20168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:30.614519 20168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:30.614580 20168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.614624 20168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:30.614652 20168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:30.620361 20168 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:46379
I20260925 12:25:30.620450 20295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:46379 every 8 connection(s)
I20260925 12:25:30.620808 20168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:30.624671 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20168
I20260925 12:25:30.624774 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:30.624946 1214 external_mini_cluster.cc:1428] Setting key b12763d96752b3caedf24af834a432e3
I20260925 12:25:30.626179 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.621643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:43209 (local address 127.1.47.190:37983)
0925 12:25:30.621730 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.621732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.621931 (+ 199us) server_negotiation.cc:407] Connection header received
0925 12:25:30.622420 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.622421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.622431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.622480 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:30.623139 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.623625 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.624609 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.624928 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.625006 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.625851 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:30.625852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:30.625852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:30.625857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:30.625892 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:30.625896 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:30.625962 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.626071 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.626099 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 12:25:30.626401 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.621600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:37983 (local address 127.1.47.131:43209)
0925 12:25:30.621809 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.621847 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.622325 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.622544 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.622547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.622555 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:30.623070 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.623075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.623706 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.623707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.624527 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.624529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.625030 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.625031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.625170 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.625721 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:30.625721 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:30.625750 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:30.625772 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:30.625965 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:30.625977 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:30.625981 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.626073 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.626181 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260925 12:25:30.626796 20296 heartbeater.cc:347] Connected to a master server at 127.1.47.190:37983
I20260925 12:25:30.626904 20296 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:30.627086 20296 heartbeater.cc:511] Master 127.1.47.190:37983 requested a full tablet report, sending...
I20260925 12:25:30.627545 19845 ts_manager.cc:194] Registered new tserver with Master: 708104f012e1499999ba6ae201bfe593 (127.1.47.131:46379)
I20260925 12:25:30.627990 19845 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.131:43209
I20260925 12:25:30.637177 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:30.641525 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.638579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.1.47.190:37983)
0925 12:25:30.638655 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.638657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.638715 (+ 58us) server_negotiation.cc:407] Connection header received
0925 12:25:30.638755 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.638756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.638765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.638808 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:30.638964 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.639438 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.640227 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.640414 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.640468 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.641161 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:30.641162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:30.641162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:30.641175 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:30.641189 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:30.641192 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:30.641281 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.641382 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.641420 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 12:25:30.643126 19845 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:30.644881 19845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:30.651312 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:34063 (local address 127.0.0.1:47598)
0925 12:25:30.647711 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.647719 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.647758 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.648032 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.648033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.648034 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:30.648065 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.648066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.648909 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.648910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.649894 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.649897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.651016 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.651017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.651090 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.651102 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.651166 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.651228 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":23,"thread_start_us":108,"threads_started":1}
I20260925 12:25:30.651584 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43705 (local address 127.0.0.1:46628)
0925 12:25:30.647541 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.647552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.647610 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.647941 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.647944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.647947 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:30.648031 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.648032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.648966 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.648967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.650170 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.650172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.651311 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.651313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.651392 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.651396 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.651450 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.651489 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":25,"thread_start_us":126,"threads_started":1}
I20260925 12:25:30.651659 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.1.47.130:34063)
0925 12:25:30.647630 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.647633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.647815 (+ 182us) server_negotiation.cc:407] Connection header received
0925 12:25:30.647882 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.647885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.647899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.647968 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:30.648138 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.648826 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.649999 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.650822 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.650890 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.650982 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.651533 (+ 551us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.651576 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260925 12:25:30.651926 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46628 (local address 127.1.47.129:43705)
0925 12:25:30.647593 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.647598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.647610 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:25:30.647733 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.647736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.647753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.647829 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:30.648136 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.648847 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.650262 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.651083 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.651165 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.651272 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.651752 (+ 480us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.651807 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260925 12:25:30.652467 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.1.47.131:46379)
0925 12:25:30.647683 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.647688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.647698 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:30.647742 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.647744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.647762 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.647836 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:30.648957 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.650041 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.651256 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.652161 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.652226 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.652314 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.652377 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.652409 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":76,"threads_started":1}
I20260925 12:25:30.652475 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.647146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46379 (local address 127.0.0.1:35712)
0925 12:25:30.647353 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.647397 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.647463 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.647941 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.647948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.647951 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:30.648035 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.648037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.650125 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.650126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.651173 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.651175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.652226 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.652228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.652290 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.652295 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.652356 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.652409 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":7,"thread_start_us":60,"threads_started":1}
I20260925 12:25:30.652402 20099 tablet_service.cc:1467] Processing CreateTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f (DEFAULT_TABLE table=test-table [id=69f54923ab1746c2a593d87328d8574b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:30.652781 20099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7e46948f9c43f1a042e1f5f1cac63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:30.653126 20230 tablet_service.cc:1467] Processing CreateTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f (DEFAULT_TABLE table=test-table [id=69f54923ab1746c2a593d87328d8574b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:30.653367 20230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7e46948f9c43f1a042e1f5f1cac63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:30.655349 19968 tablet_service.cc:1467] Processing CreateTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f (DEFAULT_TABLE table=test-table [id=69f54923ab1746c2a593d87328d8574b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:30.655637 20310 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Bootstrap starting.
I20260925 12:25:30.655685 19968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7e46948f9c43f1a042e1f5f1cac63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:30.656054 20311 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Bootstrap starting.
I20260925 12:25:30.656322 20310 tablet_bootstrap.cc:654] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:30.656654 20310 log.cc:824] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:30.656695 20311 tablet_bootstrap.cc:654] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:30.657006 20311 log.cc:824] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:30.657544 20310 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: No bootstrap required, opened a new log
I20260925 12:25:30.657622 20310 ts_tablet_manager.cc:1397] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 12:25:30.657749 20311 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: No bootstrap required, opened a new log
I20260925 12:25:30.657817 20311 ts_tablet_manager.cc:1397] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:25:30.658561 20314 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Bootstrap starting.
I20260925 12:25:30.659256 20314 tablet_bootstrap.cc:654] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:30.659556 20314 log.cc:824] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:30.660264 20314 tablet_bootstrap.cc:492] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: No bootstrap required, opened a new log
I20260925 12:25:30.660355 20314 ts_tablet_manager.cc:1397] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:25:30.660436 20311 raft_consensus.cc:348] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.660620 20311 raft_consensus.cc:374] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:30.660655 20311 raft_consensus.cc:729] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 708104f012e1499999ba6ae201bfe593, State: Initialized, Role: FOLLOWER
I20260925 12:25:30.660778 20311 consensus_queue.cc:260] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.660880 20310 raft_consensus.cc:348] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.661095 20296 heartbeater.cc:503] Master 127.1.47.190:37983 was elected leader, sending a full tablet report...
I20260925 12:25:30.661124 20310 raft_consensus.cc:374] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:30.661211 20310 raft_consensus.cc:729] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c341ecad76341c09e0bde092d56a7c4, State: Initialized, Role: FOLLOWER
I20260925 12:25:30.661054 20311 ts_tablet_manager.cc:1428] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 12:25:30.661387 20310 consensus_queue.cc:260] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.661669 20310 ts_tablet_manager.cc:1428] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260925 12:25:30.661686 20165 heartbeater.cc:503] Master 127.1.47.190:37983 was elected leader, sending a full tablet report...
I20260925 12:25:30.662874 20314 raft_consensus.cc:348] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.663022 20314 raft_consensus.cc:374] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:30.663061 20314 raft_consensus.cc:729] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9fa2be4f064abd88e4dce1fc1f43ea, State: Initialized, Role: FOLLOWER
I20260925 12:25:30.663189 20314 consensus_queue.cc:260] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.663453 20314 ts_tablet_manager.cc:1428] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:25:30.663501 20034 heartbeater.cc:503] Master 127.1.47.190:37983 was elected leader, sending a full tablet report...
W20260925 12:25:30.739208 20166 tablet.cc:2369] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:30.830911 20318 raft_consensus.cc:482] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:30.831085 20318 raft_consensus.cc:504] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.831491 20318 leader_election.cc:288] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c341ecad76341c09e0bde092d56a7c4 (127.1.47.130:34063), 708104f012e1499999ba6ae201bfe593 (127.1.47.131:46379)
I20260925 12:25:30.834424 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.831836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:51995 (local address 127.1.47.131:46379)
0925 12:25:30.831908 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.831909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.832000 (+ 91us) server_negotiation.cc:407] Connection header received
0925 12:25:30.832028 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.832029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.832038 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.832092 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:30.832274 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.832747 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.833676 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.834053 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.834115 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.834121 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.834296 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.834341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 12:25:30.834432 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.831679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:34063 (local address 127.1.47.129:55683)
0925 12:25:30.831735 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.831747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.831790 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.831995 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.831996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.831998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:30.832076 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.832078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.832750 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.832751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.833614 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.833617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.834197 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.834198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.834246 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.834248 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.834323 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.834360 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":8}
I20260925 12:25:30.834455 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.831759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:55683 (local address 127.1.47.130:34063)
0925 12:25:30.831814 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.831816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.831826 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:30.831893 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.831894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.831906 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.831972 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:30.832152 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.832669 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.833735 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.834086 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.834177 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.834184 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.834347 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.834389 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 12:25:30.834437 20319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.831710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:46379 (local address 127.1.47.129:51995)
0925 12:25:30.831895 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.831905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:30.831962 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:30.832106 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:30.832125 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:30.832127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:30.832200 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.832202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.832826 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.832827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.833586 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:30.833588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.834144 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.834145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:30.834216 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.834220 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:30.834269 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 12:25:30.834320 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260925 12:25:30.834789 20250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7e46948f9c43f1a042e1f5f1cac63f" candidate_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708104f012e1499999ba6ae201bfe593" is_pre_election: true
I20260925 12:25:30.834792 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7e46948f9c43f1a042e1f5f1cac63f" candidate_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c341ecad76341c09e0bde092d56a7c4" is_pre_election: true
I20260925 12:25:30.834975 20250 raft_consensus.cc:2454] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a9fa2be4f064abd88e4dce1fc1f43ea in term 0.
I20260925 12:25:30.834975 20119 raft_consensus.cc:2454] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a9fa2be4f064abd88e4dce1fc1f43ea in term 0.
I20260925 12:25:30.835206 19923 leader_election.cc:302] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9fa2be4f064abd88e4dce1fc1f43ea, 708104f012e1499999ba6ae201bfe593; no voters:
I20260925 12:25:30.835352 20318 raft_consensus.cc:2781] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:30.835412 20318 raft_consensus.cc:482] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:30.835444 20318 raft_consensus.cc:3037] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:30.836190 20318 raft_consensus.cc:504] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.836346 20318 leader_election.cc:288] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [CANDIDATE]: Term 1 election: Requested vote from peers 1c341ecad76341c09e0bde092d56a7c4 (127.1.47.130:34063), 708104f012e1499999ba6ae201bfe593 (127.1.47.131:46379)
I20260925 12:25:30.836503 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7e46948f9c43f1a042e1f5f1cac63f" candidate_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c341ecad76341c09e0bde092d56a7c4"
I20260925 12:25:30.836524 20250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7e46948f9c43f1a042e1f5f1cac63f" candidate_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708104f012e1499999ba6ae201bfe593"
I20260925 12:25:30.836582 20250 raft_consensus.cc:3037] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:30.836585 20119 raft_consensus.cc:3037] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:30.837356 20250 raft_consensus.cc:2454] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a9fa2be4f064abd88e4dce1fc1f43ea in term 1.
I20260925 12:25:30.837378 20119 raft_consensus.cc:2454] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a9fa2be4f064abd88e4dce1fc1f43ea in term 1.
I20260925 12:25:30.837569 19924 leader_election.cc:302] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9fa2be4f064abd88e4dce1fc1f43ea, 1c341ecad76341c09e0bde092d56a7c4; no voters:
I20260925 12:25:30.837666 20318 raft_consensus.cc:2781] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:30.837822 20318 raft_consensus.cc:686] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [term 1 LEADER]: Becoming Leader. State: Replica: 0a9fa2be4f064abd88e4dce1fc1f43ea, State: Running, Role: LEADER
I20260925 12:25:30.837949 20318 consensus_queue.cc:237] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } }
I20260925 12:25:30.838869 19845 catalog_manager.cc:5495] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a9fa2be4f064abd88e4dce1fc1f43ea (127.1.47.129). New cstate: current_term: 1 leader_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a9fa2be4f064abd88e4dce1fc1f43ea" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 } health_report { overall_health: UNKNOWN } } }
W20260925 12:25:30.856698 20035 tablet.cc:2369] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:25:30.871435 20297 tablet.cc:2369] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:30.877868 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.875513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.1.47.129:43705)
0925 12:25:30.875576 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:30.875578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:30.875589 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:25:30.875648 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:30.875648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:30.875670 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:30.875736 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:30.875925 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.876399 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.877146 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:30.877325 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:30.877387 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:30.877514 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:30.877603 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:30.877673 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:30.877719 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 12:25:30.877729 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 12:25:30.879802 20119 raft_consensus.cc:1261] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 1 FOLLOWER]: Refusing update from remote peer 0a9fa2be4f064abd88e4dce1fc1f43ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:30.879822 20250 raft_consensus.cc:1261] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 1 FOLLOWER]: Refusing update from remote peer 0a9fa2be4f064abd88e4dce1fc1f43ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:30.880303 20318 consensus_queue.cc:1035] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "708104f012e1499999ba6ae201bfe593" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:30.880494 20320 consensus_queue.cc:1035] T df7e46948f9c43f1a042e1f5f1cac63f P 0a9fa2be4f064abd88e4dce1fc1f43ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c341ecad76341c09e0bde092d56a7c4" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:30.882917 20323 mvcc.cc:205] Tried to move back new op lower bound from 7333229080080535552 to 7333229079912927232. Current Snapshot: MvccSnapshot[applied={T|T < 7333229080080535552}]
I20260925 12:25:30.883065 20325 mvcc.cc:205] Tried to move back new op lower bound from 7333229080080535552 to 7333229079912927232. Current Snapshot: MvccSnapshot[applied={T|T < 7333229080080535552}]
I20260925 12:25:30.883122 20324 mvcc.cc:205] Tried to move back new op lower bound from 7333229080080535552 to 7333229079912927232. Current Snapshot: MvccSnapshot[applied={T|T < 7333229080080535552}]
I20260925 12:25:30.885861 19846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:30.885990 19846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:30.886852 19846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0dbc0bde1fd4b9382f99346402a6146: Sending DeleteTablet for 3 replicas of tablet df7e46948f9c43f1a042e1f5f1cac63f
I20260925 12:25:30.887217 20230 tablet_service.cc:1514] Processing DeleteTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='slave'} at 127.0.0.1:35712
I20260925 12:25:30.887312 20099 tablet_service.cc:1514] Processing DeleteTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='slave'} at 127.0.0.1:47598
I20260925 12:25:30.887305 19968 tablet_service.cc:1514] Processing DeleteTablet for tablet df7e46948f9c43f1a042e1f5f1cac63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:30 UTC) from {username='slave'} at 127.0.0.1:46628
I20260925 12:25:30.887547 20333 tablet_replica.cc:330] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: stopping tablet replica
I20260925 12:25:30.887809 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19906
I20260925 12:25:30.888443 20333 raft_consensus.cc:2229] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:30.888564 20333 raft_consensus.cc:2258] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:30.889245 20333 ts_tablet_manager.cc:1905] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:30.889606 20335 tablet_replica.cc:330] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: stopping tablet replica
I20260925 12:25:30.889725 20335 raft_consensus.cc:2229] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:30.889834 20335 raft_consensus.cc:2258] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:30.890321 20335 ts_tablet_manager.cc:1905] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:30.891163 20333 ts_tablet_manager.cc:1918] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:30.891242 20333 log.cc:1196] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/df7e46948f9c43f1a042e1f5f1cac63f
I20260925 12:25:30.891489 20333 ts_tablet_manager.cc:1939] T df7e46948f9c43f1a042e1f5f1cac63f P 708104f012e1499999ba6ae201bfe593: Deleting consensus metadata
I20260925 12:25:30.891844 20335 ts_tablet_manager.cc:1918] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:30.891942 20335 log.cc:1196] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/df7e46948f9c43f1a042e1f5f1cac63f
I20260925 12:25:30.891970 19832 catalog_manager.cc:4841] TS 708104f012e1499999ba6ae201bfe593 (127.1.47.131:46379): tablet df7e46948f9c43f1a042e1f5f1cac63f (table test-table [id=69f54923ab1746c2a593d87328d8574b]) successfully deleted
I20260925 12:25:30.892201 20335 ts_tablet_manager.cc:1939] T df7e46948f9c43f1a042e1f5f1cac63f P 1c341ecad76341c09e0bde092d56a7c4: Deleting consensus metadata
I20260925 12:25:30.892887 19833 catalog_manager.cc:4841] TS 1c341ecad76341c09e0bde092d56a7c4 (127.1.47.130:34063): tablet df7e46948f9c43f1a042e1f5f1cac63f (table test-table [id=69f54923ab1746c2a593d87328d8574b]) successfully deleted
W20260925 12:25:30.894130 19830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43705 (error 108)
I20260925 12:25:30.894331 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20037
I20260925 12:25:30.896123 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:30.895964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43705 (local address 127.0.0.1:46636)
0925 12:25:30.896025 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:30.896079 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:25:30.896265 19830 catalog_manager.cc:4584] TS 0a9fa2be4f064abd88e4dce1fc1f43ea (127.1.47.129:43705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df7e46948f9c43f1a042e1f5f1cac63f: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43705: connect: Connection refused (error 111)
I20260925 12:25:30.899314 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20168
I20260925 12:25:30.904177 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 19815
2026-09-25T12:25:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (860 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:30 dist-test-slave-l9ww krb5kdc[20339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:30 dist-test-slave-l9ww krb5kdc[20339](info): set up 2 sockets
Sep 25 12:25:30 dist-test-slave-l9ww krb5kdc[20339](info): commencing operation
krb5kdc: starting...
W20260925 12:25:31.935798 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:32.058727 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36921
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:32.141659 20352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:32.141943 20352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:32.141981 20352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:32.143193 20352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:32.143273 20352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:32.143297 20352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:32.143318 20352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:32.143352 20352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:32.144924 20352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:32.145234 20352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:32.145481 20352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:32.150627 20357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.150612 20360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.150661 20352 server_base.cc:1034] running on GCE node
W20260925 12:25:32.150671 20358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.151111 20352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:32.151433 20352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:32.152598 20352 hybrid_clock.cc:648] HybridClock initialized: now 1790339132152575 us; error 40 us; skew 500 ppm
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:32.155244 20352 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:32.155751 20352 webserver.cc:460] Webserver started at http://127.1.47.190:33061/ using document root <none> and password file <none>
I20260925 12:25:32.156029 20352 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:32.156100 20352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:32.156265 20352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:32.157357 20352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "03ce5f97a6d3487fa84327e900f80851"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "8b93c437c50e61040642900628f16c54"
server_key_iv: "bf3b27525e5f670ee5cbe32f401cadae"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.157740 20352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "03ce5f97a6d3487fa84327e900f80851"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "8b93c437c50e61040642900628f16c54"
server_key_iv: "bf3b27525e5f670ee5cbe32f401cadae"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.159066 20352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:32.159749 20367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.159955 20352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:32.160058 20352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "03ce5f97a6d3487fa84327e900f80851"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "8b93c437c50e61040642900628f16c54"
server_key_iv: "bf3b27525e5f670ee5cbe32f401cadae"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.160142 20352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:32.184955 20352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:32.187111 20352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:32.187346 20352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:32.191563 20352 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:36921
I20260925 12:25:32.191594 20419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:36921 every 8 connection(s)
I20260925 12:25:32.192004 20352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:32.192883 20420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:32.194089 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20352
I20260925 12:25:32.194234 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:32.194434 1214 external_mini_cluster.cc:1428] Setting key a1b9ee1def244b2e2c68ba2c02db467e
I20260925 12:25:32.195482 20420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: Bootstrap starting.
I20260925 12:25:32.196223 20420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:32.196563 20420 log.cc:824] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:32.197348 20420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: No bootstrap required, opened a new log
I20260925 12:25:32.199563 20420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } }
I20260925 12:25:32.199760 20420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:32.199848 20420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ce5f97a6d3487fa84327e900f80851, State: Initialized, Role: FOLLOWER
I20260925 12:25:32.200060 20420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } }
I20260925 12:25:32.200152 20420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:32.200191 20420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:32.200232 20420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:32.200799 20420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } }
I20260925 12:25:32.200927 20420 leader_election.cc:302] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ce5f97a6d3487fa84327e900f80851; no voters:
I20260925 12:25:32.201134 20420 leader_election.cc:288] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:32.201165 20425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:32.201277 20425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [term 1 LEADER]: Becoming Leader. State: Replica: 03ce5f97a6d3487fa84327e900f80851, State: Running, Role: LEADER
I20260925 12:25:32.201371 20425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } }
I20260925 12:25:32.201411 20420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:32.201701 20426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ce5f97a6d3487fa84327e900f80851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } } }
I20260925 12:25:32.201810 20426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [sys.catalog]: This master's current role is: LEADER
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:32.201699 20427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ce5f97a6d3487fa84327e900f80851. Latest consensus state: current_term: 1 leader_uuid: "03ce5f97a6d3487fa84327e900f80851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ce5f97a6d3487fa84327e900f80851" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36921 } } }
I20260925 12:25:32.202039 20427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:32.202560 20431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:32.203312 20431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:32.204916 20431 catalog_manager.cc:1347] Generated new cluster ID: b9f40e5b520d408991a47b7e107de649
I20260925 12:25:32.204972 20431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:32.206715 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.195085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49460 (local address 127.1.47.190:36921)
0925 12:25:32.195322 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.195330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.195357 (+ 27us) server_negotiation.cc:407] Connection header received
0925 12:25:32.195847 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.195910 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.195977 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.196105 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:32.196928 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.197957 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.199051 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.199456 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.199532 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.202501 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:32.202506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:32.202506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:32.202521 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:32.204650 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.205113 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.205116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.205116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.205221 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.205405 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.205405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.205406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.205687 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:32.205830 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.206358 (+ 528us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.206498 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":137,"threads_started":1}
I20260925 12:25:32.226475 20431 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:32.227051 20431 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:32.230975 20431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: Generated new TSK 0
I20260925 12:25:32.231235 20431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
Entry for principal kudu/127.1.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab.
I20260925 12:25:32.262620 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:32.351521 20446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:32.351747 20446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:32.351771 20446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:32.352980 20446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:32.353030 20446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:32.353058 20446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:32.354472 20446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.129.keytab
--principal=kudu/127.1.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:32.354669 20446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:32.354852 20446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:32.360132 20454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.360127 20451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.360127 20452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.360455 20446 server_base.cc:1034] running on GCE node
I20260925 12:25:32.360682 20446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:32.360935 20446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:32.362105 20446 hybrid_clock.cc:648] HybridClock initialized: now 1790339132362096 us; error 30 us; skew 500 ppm
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:32.364784 20446 init.cc:377] Logged in from keytab as kudu/127.1.47.129@KRBTEST.COM (short username kudu)
I20260925 12:25:32.365288 20446 webserver.cc:460] Webserver started at http://127.1.47.129:33891/ using document root <none> and password file <none>
I20260925 12:25:32.365523 20446 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:32.365590 20446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:32.365744 20446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:32.366812 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "df76c929add843eb81a08a8be188ece9"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "6649d2835b4730d0b9dbaaed5575b7f3"
server_key_iv: "e16afab9dd6409a9adb7412771122afc"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.367177 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "df76c929add843eb81a08a8be188ece9"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "6649d2835b4730d0b9dbaaed5575b7f3"
server_key_iv: "e16afab9dd6409a9adb7412771122afc"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.368657 20446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:32.369499 20461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.369714 20446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:32.369815 20446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "df76c929add843eb81a08a8be188ece9"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "6649d2835b4730d0b9dbaaed5575b7f3"
server_key_iv: "e16afab9dd6409a9adb7412771122afc"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.369901 20446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:32.389804 20446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:32.392270 20446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:32.392457 20446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:32.392715 20446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:32.393057 20446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:32.393114 20446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.393149 20446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:32.393178 20446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.399300 20446 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:36463
I20260925 12:25:32.399356 20574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:36463 every 8 connection(s)
I20260925 12:25:32.399729 20446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.129@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:32.408720 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20446
I20260925 12:25:32.408809 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:32.409011 1214 external_mini_cluster.cc:1428] Setting key 4c63f8a9716d1afa93f180c77f5f9dd9
I20260925 12:25:32.409126 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.400544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:48119 (local address 127.1.47.190:36921)
0925 12:25:32.400618 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.400620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.400874 (+ 254us) server_negotiation.cc:407] Connection header received
0925 12:25:32.401395 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.401396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.401407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.401472 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:32.402478 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.403057 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.404098 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.404297 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.404352 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.406478 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:32.406479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:32.406479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:32.406485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:32.407654 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.408130 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.408131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.408131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.408185 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.408420 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.408421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.408421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.408523 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:32.408607 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.408940 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.409034 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:25:32.409346 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.400514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36921 (local address 127.1.47.129:48119)
0925 12:25:32.400708 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.400745 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.401296 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.401545 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.401548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.402136 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:32.402404 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.402410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.403137 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.403138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.404028 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.404030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.404371 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.404371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.404471 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.405021 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:32.405029 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:32.406387 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:32.407725 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.407731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.407733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.408077 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.408248 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.408254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.408254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.408360 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.408600 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:32.408607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:32.408757 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.408916 (+ 159us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.409080 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
I20260925 12:25:32.409871 20575 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36921
I20260925 12:25:32.409982 20575 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:32.410175 20575 heartbeater.cc:511] Master 127.1.47.190:36921 requested a full tablet report, sending...
I20260925 12:25:32.410720 20383 ts_manager.cc:194] Registered new tserver with Master: df76c929add843eb81a08a8be188ece9 (127.1.47.129:36463)
I20260925 12:25:32.411633 20383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.129@KRBTEST.COM'} at 127.1.47.129:48119
WARNING: no policy specified for kudu/127.1.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
Entry for principal kudu/127.1.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab.
I20260925 12:25:32.438416 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:32.523291 20580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:32.523597 20580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:32.523649 20580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:32.525025 20580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:32.525106 20580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:32.525153 20580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:32.526793 20580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.130.keytab
--principal=kudu/127.1.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:32.527088 20580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:32.527359 20580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:32.532949 20586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.533032 20580 server_base.cc:1034] running on GCE node
W20260925 12:25:32.532929 20588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.532929 20585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.533476 20580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:32.533686 20580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:32.534822 20580 hybrid_clock.cc:648] HybridClock initialized: now 1790339132534805 us; error 42 us; skew 500 ppm
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:32.537464 20580 init.cc:377] Logged in from keytab as kudu/127.1.47.130@KRBTEST.COM (short username kudu)
I20260925 12:25:32.537861 20580 webserver.cc:460] Webserver started at http://127.1.47.130:40935/ using document root <none> and password file <none>
I20260925 12:25:32.538094 20580 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:32.538162 20580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:32.538316 20580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:32.539397 20580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "e7accc4626054d02ae1f26c2aff9f679"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "444c2e89106a7cd63d8d236d0ec7b376"
server_key_iv: "ca5b8ef36f08f20c23b70975d92d05d1"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.539764 20580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "e7accc4626054d02ae1f26c2aff9f679"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "444c2e89106a7cd63d8d236d0ec7b376"
server_key_iv: "ca5b8ef36f08f20c23b70975d92d05d1"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.541174 20580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:32.541967 20595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.542157 20580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:32.542254 20580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "e7accc4626054d02ae1f26c2aff9f679"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "444c2e89106a7cd63d8d236d0ec7b376"
server_key_iv: "ca5b8ef36f08f20c23b70975d92d05d1"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.542337 20580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:32.566687 20580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:32.568681 20580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:32.568851 20580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:32.569099 20580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:32.569416 20580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:32.569471 20580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.569512 20580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:32.569535 20580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.575698 20580 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:35659
I20260925 12:25:32.575901 20708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:35659 every 8 connection(s)
I20260925 12:25:32.576182 20580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.130@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:32.583506 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20580
I20260925 12:25:32.583634 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:32.583905 1214 external_mini_cluster.cc:1428] Setting key 6e6604a33a4056fc17a7094724ed995c
I20260925 12:25:32.586485 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.576956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:44971 (local address 127.1.47.190:36921)
0925 12:25:32.577006 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.577009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.577280 (+ 271us) server_negotiation.cc:407] Connection header received
0925 12:25:32.577785 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.577786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.577797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.577862 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:32.578772 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.579268 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.580412 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.580595 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.580649 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.582678 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:32.582680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:32.582680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:32.582687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:32.584604 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.585187 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.585188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.585189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.585276 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.585596 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.585597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.585598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.585803 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:32.585929 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.586270 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.586383 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:25:32.587076 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.576953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36921 (local address 127.1.47.130:44971)
0925 12:25:32.577146 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.577186 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.577675 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.577922 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.577925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.578499 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:32.578695 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.578699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.579346 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.579347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.580341 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.580344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.580667 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.580668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.580765 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.581220 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:32.581228 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:32.582583 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:32.584696 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.584703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.584705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.585091 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.585362 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.585363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.585363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.585516 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.585907 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:32.585913 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:32.586089 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.586582 (+ 493us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.586810 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260925 12:25:32.587615 20709 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36921
I20260925 12:25:32.587711 20709 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:32.587924 20709 heartbeater.cc:511] Master 127.1.47.190:36921 requested a full tablet report, sending...
I20260925 12:25:32.588366 20383 ts_manager.cc:194] Registered new tserver with Master: e7accc4626054d02ae1f26c2aff9f679 (127.1.47.130:35659)
I20260925 12:25:32.588817 20383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.130@KRBTEST.COM'} at 127.1.47.130:44971
WARNING: no policy specified for kudu/127.1.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
Entry for principal kudu/127.1.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab.
I20260925 12:25:32.612905 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:32.694453 20714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:32.694665 20714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:32.694689 20714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:32.695914 20714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:32.695966 20714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:32.695994 20714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:32.697460 20714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.131.keytab
--principal=kudu/127.1.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:36921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:32.697690 20714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:32.697934 20714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:32.703109 20720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.703109 20722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:32.703145 20719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:32.703620 20714 server_base.cc:1034] running on GCE node
I20260925 12:25:32.703830 20714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:32.704120 20714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:32.705261 20714 hybrid_clock.cc:648] HybridClock initialized: now 1790339132705252 us; error 50 us; skew 500 ppm
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:32.707720 20714 init.cc:377] Logged in from keytab as kudu/127.1.47.131@KRBTEST.COM (short username kudu)
I20260925 12:25:32.708220 20714 webserver.cc:460] Webserver started at http://127.1.47.131:39323/ using document root <none> and password file <none>
I20260925 12:25:32.708456 20714 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:32.708503 20714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:32.708669 20714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:32.709719 20714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "40f0c3ef8e664b8199cbbae2b154c735"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "2ac217c1219f48bd70462443fcf49e39"
server_key_iv: "1e7ad634a7deab5aef07cdf4f8ee9744"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.710079 20714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "40f0c3ef8e664b8199cbbae2b154c735"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "2ac217c1219f48bd70462443fcf49e39"
server_key_iv: "1e7ad634a7deab5aef07cdf4f8ee9744"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.711388 20714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:32.712144 20729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.712334 20714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:32.712404 20714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "40f0c3ef8e664b8199cbbae2b154c735"
format_stamp: "Formatted at 2026-09-25 12:25:32 on dist-test-slave-l9ww"
server_key: "2ac217c1219f48bd70462443fcf49e39"
server_key_iv: "1e7ad634a7deab5aef07cdf4f8ee9744"
server_key_version: "encryptionkey@0"
I20260925 12:25:32.712495 20714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:32.756544 20714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:32.759138 20714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:32.759297 20714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:32.759546 20714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:32.759948 20714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:32.760005 20714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.760048 20714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:32.760075 20714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:32.765990 20714 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:45137
I20260925 12:25:32.766031 20842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:45137 every 8 connection(s)
I20260925 12:25:32.766356 20714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:32.769520 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20714
I20260925 12:25:32.769624 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:32.769816 1214 external_mini_cluster.cc:1428] Setting key 00e83deb0bb562975a6c0e69d6deb413
Sep 25 12:25:32 dist-test-slave-l9ww krb5kdc[20339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339132, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.131@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:32.775759 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.767115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:56367 (local address 127.1.47.190:36921)
0925 12:25:32.767211 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.767213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.767329 (+ 116us) server_negotiation.cc:407] Connection header received
0925 12:25:32.767822 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.767824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.767835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.767963 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:32.768908 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.769477 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.770766 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.770960 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.771018 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.773208 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:32.773210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:32.773210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:32.773216 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:32.774384 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.774725 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.774726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.774726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.774801 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.775113 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.775114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.775114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.775233 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:32.775305 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.775581 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.775659 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 12:25:32.775975 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.767037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:36921 (local address 127.1.47.131:56367)
0925 12:25:32.767201 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.767241 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.767708 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.768015 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.768019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.768628 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:32.768825 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.768830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.769568 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.769569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.770694 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.770696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.771029 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.771030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.771124 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.771647 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:32.771655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:32.773117 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:32.774452 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.774457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.774459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.774663 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.774889 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:32.774890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:32.774891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:32.775033 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:32.775307 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:32.775314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:32.775452 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.775560 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.775716 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260925 12:25:32.776383 20843 heartbeater.cc:347] Connected to a master server at 127.1.47.190:36921
I20260925 12:25:32.776497 20843 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:32.776678 20843 heartbeater.cc:511] Master 127.1.47.190:36921 requested a full tablet report, sending...
I20260925 12:25:32.777069 20383 ts_manager.cc:194] Registered new tserver with Master: 40f0c3ef8e664b8199cbbae2b154c735 (127.1.47.131:45137)
I20260925 12:25:32.777452 20383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.47.131@KRBTEST.COM'} at 127.1.47.131:56367
I20260925 12:25:32.781487 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:32.788189 20423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.782742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.1.47.190:36921)
0925 12:25:32.782793 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.782795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.782875 (+ 80us) server_negotiation.cc:407] Connection header received
0925 12:25:32.782914 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.782915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.782922 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.782972 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:32.783498 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.784012 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.784681 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.784857 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.784904 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.786035 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:32.786036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:32.786036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:32.786041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:32.787165 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.787473 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.787474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.787474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.787519 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:32.787673 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:32.787673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:32.787673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:32.787810 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:32.787943 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.788065 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.788127 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 12:25:32.789656 20383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:32.791145 20383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:32.797477 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45137 (local address 127.0.0.1:40414)
0925 12:25:32.793879 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.793888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.793937 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.794194 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.794195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.794197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:32.794251 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.794253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795168 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.795169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.796507 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.796510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797193 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.797267 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797281 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.797349 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.797400 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":28,"thread_start_us":70,"threads_started":1}
I20260925 12:25:32.797487 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.1.47.131:45137)
0925 12:25:32.793880 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.793885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.793926 (+ 41us) server_negotiation.cc:407] Connection header received
0925 12:25:32.794019 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.794022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.794037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.794113 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:32.794331 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795066 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.796605 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797105 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797194 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797255 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.797362 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.797400 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260925 12:25:32.797670 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36463 (local address 127.0.0.1:60124)
0925 12:25:32.793665 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.793706 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.793772 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.794357 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.794358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.794360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:32.794395 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.794396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795589 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.795590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.796496 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.796498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797430 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.797497 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797500 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.797537 (+ 37us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.797585 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":90,"threads_started":1}
I20260925 12:25:32.797833 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.1.47.130:35659)
0925 12:25:32.793906 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.793911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.793921 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:32.793963 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.793965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.793981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.794061 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:32.794222 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795379 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.796639 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797315 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797382 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797439 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.797712 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.797760 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":72,"threads_started":1}
I20260925 12:25:32.797672 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:35659 (local address 127.0.0.1:48064)
0925 12:25:32.793668 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:32.793706 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:32.793770 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:32.794060 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:32.794063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:32.794064 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:32.794135 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.794137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795482 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.795483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.796575 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:32.796577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797466 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:32.797522 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797531 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 12:25:32.797576 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 12:25:32.797603 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":47,"threads_started":1}
I20260925 12:25:32.798053 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:32.793458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.1.47.129:36463)
0925 12:25:32.794198 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:32.794202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:32.794212 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:32.794248 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:32.794250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:32.794268 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:32.794339 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:32.794448 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.795492 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.796580 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:32.797307 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:32.797799 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:32.797868 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:32.797894 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 12:25:32.797944 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":106,"threads_started":1}
I20260925 12:25:32.798318 20777 tablet_service.cc:1467] Processing CreateTablet for tablet 585c6e0464c24dc9979092c534093529 (DEFAULT_TABLE table=test-table [id=ae6310682087460fa6bfdee7378f1b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:32.798686 20777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c6e0464c24dc9979092c534093529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:32.798623 20509 tablet_service.cc:1467] Processing CreateTablet for tablet 585c6e0464c24dc9979092c534093529 (DEFAULT_TABLE table=test-table [id=ae6310682087460fa6bfdee7378f1b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:32.798929 20509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c6e0464c24dc9979092c534093529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:32.800225 20643 tablet_service.cc:1467] Processing CreateTablet for tablet 585c6e0464c24dc9979092c534093529 (DEFAULT_TABLE table=test-table [id=ae6310682087460fa6bfdee7378f1b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:32.800480 20643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c6e0464c24dc9979092c534093529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:32.802214 20861 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Bootstrap starting.
I20260925 12:25:32.802973 20861 tablet_bootstrap.cc:654] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:32.803561 20861 log.cc:824] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:32.803611 20862 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Bootstrap starting.
I20260925 12:25:32.803943 20860 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Bootstrap starting.
I20260925 12:25:32.804267 20862 tablet_bootstrap.cc:654] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:32.804607 20860 tablet_bootstrap.cc:654] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:32.804827 20861 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: No bootstrap required, opened a new log
I20260925 12:25:32.804932 20861 ts_tablet_manager.cc:1397] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 12:25:32.804978 20862 log.cc:824] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:32.805199 20860 log.cc:824] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:32.805812 20862 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: No bootstrap required, opened a new log
I20260925 12:25:32.805902 20862 ts_tablet_manager.cc:1397] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 12:25:32.806605 20861 raft_consensus.cc:348] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.806743 20861 raft_consensus.cc:374] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:32.806768 20861 raft_consensus.cc:729] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df76c929add843eb81a08a8be188ece9, State: Initialized, Role: FOLLOWER
I20260925 12:25:32.806883 20861 consensus_queue.cc:260] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.807170 20861 ts_tablet_manager.cc:1428] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:25:32.807183 20575 heartbeater.cc:503] Master 127.1.47.190:36921 was elected leader, sending a full tablet report...
I20260925 12:25:32.807775 20862 raft_consensus.cc:348] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.807999 20862 raft_consensus.cc:374] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:32.808056 20862 raft_consensus.cc:729] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7accc4626054d02ae1f26c2aff9f679, State: Initialized, Role: FOLLOWER
I20260925 12:25:32.808198 20862 consensus_queue.cc:260] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.808496 20862 ts_tablet_manager.cc:1428] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 12:25:32.808511 20709 heartbeater.cc:503] Master 127.1.47.190:36921 was elected leader, sending a full tablet report...
I20260925 12:25:32.808696 20860 tablet_bootstrap.cc:492] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: No bootstrap required, opened a new log
I20260925 12:25:32.808776 20860 ts_tablet_manager.cc:1397] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 12:25:32.810459 20860 raft_consensus.cc:348] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.810590 20860 raft_consensus.cc:374] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:32.810611 20860 raft_consensus.cc:729] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f0c3ef8e664b8199cbbae2b154c735, State: Initialized, Role: FOLLOWER
I20260925 12:25:32.810743 20860 consensus_queue.cc:260] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:32.811017 20860 ts_tablet_manager.cc:1428] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 12:25:32.811033 20843 heartbeater.cc:503] Master 127.1.47.190:36921 was elected leader, sending a full tablet report...
W20260925 12:25:32.826771 20710 tablet.cc:2369] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:25:32.900585 20576 tablet.cc:2369] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 12:25:33.016880 20844 tablet.cc:2369] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:33.120123 20866 raft_consensus.cc:482] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:33.120260 20866 raft_consensus.cc:504] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:33.120616 20866 leader_election.cc:288] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7accc4626054d02ae1f26c2aff9f679 (127.1.47.130:35659), 40f0c3ef8e664b8199cbbae2b154c735 (127.1.47.131:45137)
I20260925 12:25:33.123636 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.120881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:33907 (local address 127.1.47.130:35659)
0925 12:25:33.120928 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.120930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.121057 (+ 127us) server_negotiation.cc:407] Connection header received
0925 12:25:33.121132 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.121133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.121145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.121221 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.121409 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.121867 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.122880 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.123252 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.123342 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.123351 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.123511 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.123544 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:33.123656 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.120914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:53853 (local address 127.1.47.131:45137)
0925 12:25:33.120962 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.120964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.121054 (+ 90us) server_negotiation.cc:407] Connection header received
0925 12:25:33.121135 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.121136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.121146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.121217 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.121390 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.121862 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.122869 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.123270 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.123372 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.123380 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.123544 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.123580 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:25:33.123700 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.120810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:45137 (local address 127.1.47.129:53853)
0925 12:25:33.120968 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.120978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.121070 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.121223 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.121224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.121226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.121315 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.121317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.121952 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.121953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.122766 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.122768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.123376 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.123378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.123457 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.123461 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.123535 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.123575 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":18,"thread_start_us":70,"threads_started":1}
I20260925 12:25:33.123699 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.120812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:35659 (local address 127.1.47.129:33907)
0925 12:25:33.120969 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.120986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.121070 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.121225 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.121226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.121228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.121334 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.121336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.121954 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.121955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.122786 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.122788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.123344 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.123345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.123424 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.123428 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.123494 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.123548 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":53,"thread_start_us":65,"threads_started":1}
I20260925 12:25:33.124213 20663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c6e0464c24dc9979092c534093529" candidate_uuid: "df76c929add843eb81a08a8be188ece9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7accc4626054d02ae1f26c2aff9f679" is_pre_election: true
I20260925 12:25:33.124224 20797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c6e0464c24dc9979092c534093529" candidate_uuid: "df76c929add843eb81a08a8be188ece9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40f0c3ef8e664b8199cbbae2b154c735" is_pre_election: true
I20260925 12:25:33.124347 20663 raft_consensus.cc:2454] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df76c929add843eb81a08a8be188ece9 in term 0.
I20260925 12:25:33.124378 20797 raft_consensus.cc:2454] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df76c929add843eb81a08a8be188ece9 in term 0.
I20260925 12:25:33.124624 20465 leader_election.cc:302] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df76c929add843eb81a08a8be188ece9, e7accc4626054d02ae1f26c2aff9f679; no voters:
I20260925 12:25:33.124747 20866 raft_consensus.cc:2781] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:33.124802 20866 raft_consensus.cc:482] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:33.124838 20866 raft_consensus.cc:3037] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.125540 20866 raft_consensus.cc:504] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:33.125691 20866 leader_election.cc:288] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [CANDIDATE]: Term 1 election: Requested vote from peers e7accc4626054d02ae1f26c2aff9f679 (127.1.47.130:35659), 40f0c3ef8e664b8199cbbae2b154c735 (127.1.47.131:45137)
I20260925 12:25:33.125867 20663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c6e0464c24dc9979092c534093529" candidate_uuid: "df76c929add843eb81a08a8be188ece9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7accc4626054d02ae1f26c2aff9f679"
I20260925 12:25:33.125900 20797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c6e0464c24dc9979092c534093529" candidate_uuid: "df76c929add843eb81a08a8be188ece9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40f0c3ef8e664b8199cbbae2b154c735"
I20260925 12:25:33.125946 20663 raft_consensus.cc:3037] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.125962 20797 raft_consensus.cc:3037] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.126821 20663 raft_consensus.cc:2454] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df76c929add843eb81a08a8be188ece9 in term 1.
I20260925 12:25:33.126821 20797 raft_consensus.cc:2454] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df76c929add843eb81a08a8be188ece9 in term 1.
I20260925 12:25:33.127096 20465 leader_election.cc:302] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df76c929add843eb81a08a8be188ece9, e7accc4626054d02ae1f26c2aff9f679; no voters:
I20260925 12:25:33.127207 20866 raft_consensus.cc:2781] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:33.127415 20866 raft_consensus.cc:686] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [term 1 LEADER]: Becoming Leader. State: Replica: df76c929add843eb81a08a8be188ece9, State: Running, Role: LEADER
I20260925 12:25:33.127534 20866 consensus_queue.cc:237] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } }
I20260925 12:25:33.128492 20383 catalog_manager.cc:5495] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 reported cstate change: term changed from 0 to 1, leader changed from <none> to df76c929add843eb81a08a8be188ece9 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "df76c929add843eb81a08a8be188ece9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df76c929add843eb81a08a8be188ece9" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 36463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 } health_report { overall_health: UNKNOWN } } }
I20260925 12:25:33.150211 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.147724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.1.47.129:36463)
0925 12:25:33.147805 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.147807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.147816 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:33.147904 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.147905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.147931 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.148026 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:33.148148 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.148627 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.149412 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.149631 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.149709 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.149861 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:33.149975 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:33.150071 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.150103 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.150123 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 12:25:33.152072 20797 raft_consensus.cc:1261] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 1 FOLLOWER]: Refusing update from remote peer df76c929add843eb81a08a8be188ece9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:33.152073 20663 raft_consensus.cc:1261] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 1 FOLLOWER]: Refusing update from remote peer df76c929add843eb81a08a8be188ece9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:33.152452 20866 consensus_queue.cc:1035] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7accc4626054d02ae1f26c2aff9f679" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:33.152626 20871 consensus_queue.cc:1035] T 585c6e0464c24dc9979092c534093529 P df76c929add843eb81a08a8be188ece9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40f0c3ef8e664b8199cbbae2b154c735" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:33.155405 20876 mvcc.cc:205] Tried to move back new op lower bound from 7333229089388007424 to 7333229089291157504. Current Snapshot: MvccSnapshot[applied={T|T < 7333229089388007424}]
I20260925 12:25:33.157155 20383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:33.157272 20383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:33.157996 20383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03ce5f97a6d3487fa84327e900f80851: Sending DeleteTablet for 3 replicas of tablet 585c6e0464c24dc9979092c534093529
I20260925 12:25:33.158288 20509 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c6e0464c24dc9979092c534093529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:60124
I20260925 12:25:33.158329 20643 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c6e0464c24dc9979092c534093529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:48064
I20260925 12:25:33.158406 20777 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c6e0464c24dc9979092c534093529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='kudu', principal='kudu/127.1.47.190@KRBTEST.COM'} at 127.0.0.1:40414
I20260925 12:25:33.158617 20884 tablet_replica.cc:330] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: stopping tablet replica
I20260925 12:25:33.158655 20885 tablet_replica.cc:330] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: stopping tablet replica
I20260925 12:25:33.158723 20884 raft_consensus.cc:2229] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:33.158855 20884 raft_consensus.cc:2258] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:33.158871 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20446
I20260925 12:25:33.160195 20884 ts_tablet_manager.cc:1905] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:33.160709 20885 raft_consensus.cc:2229] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:33.160849 20885 raft_consensus.cc:2258] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:33.161288 20885 ts_tablet_manager.cc:1905] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:33.161923 20884 ts_tablet_manager.cc:1918] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:33.162007 20884 log.cc:1196] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-1/wal/wals/585c6e0464c24dc9979092c534093529
I20260925 12:25:33.162246 20884 ts_tablet_manager.cc:1939] T 585c6e0464c24dc9979092c534093529 P e7accc4626054d02ae1f26c2aff9f679: Deleting consensus metadata
I20260925 12:25:33.162681 20371 catalog_manager.cc:4841] TS e7accc4626054d02ae1f26c2aff9f679 (127.1.47.130:35659): tablet 585c6e0464c24dc9979092c534093529 (table test-table [id=ae6310682087460fa6bfdee7378f1b0f]) successfully deleted
I20260925 12:25:33.163358 20885 ts_tablet_manager.cc:1918] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:33.163434 20885 log.cc:1196] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/585c6e0464c24dc9979092c534093529
I20260925 12:25:33.163688 20885 ts_tablet_manager.cc:1939] T 585c6e0464c24dc9979092c534093529 P 40f0c3ef8e664b8199cbbae2b154c735: Deleting consensus metadata
I20260925 12:25:33.164182 20368 catalog_manager.cc:4841] TS 40f0c3ef8e664b8199cbbae2b154c735 (127.1.47.131:45137): tablet 585c6e0464c24dc9979092c534093529 (table test-table [id=ae6310682087460fa6bfdee7378f1b0f]) successfully deleted
W20260925 12:25:33.165040 20368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:36463 (error 108)
I20260925 12:25:33.165542 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.165405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:36463 (local address 127.0.0.1:60150)
0925 12:25:33.165465 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.165502 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 12:25:33.165627 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20580
W20260925 12:25:33.171202 20368 catalog_manager.cc:4584] TS df76c929add843eb81a08a8be188ece9 (127.1.47.129:36463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 585c6e0464c24dc9979092c534093529: Network error: Client connection negotiation failed: client connection to 127.1.47.129:36463: connect: Connection refused (error 111)
I20260925 12:25:33.171468 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20714
I20260925 12:25:33.177573 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20352
2026-09-25T12:25:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2274 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T12:25:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:33Z Disabled control of system clock
I20260925 12:25:33.205096 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42715
--webserver_interface=127.1.47.190
--webserver_port=0
--builtin_ntp_servers=127.1.47.148:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:33.286455 20890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:33.286692 20890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:33.286712 20890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:33.287953 20890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:33.287997 20890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:33.288009 20890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:33.288022 20890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:33.288034 20890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:33.289513 20890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:42715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:33.289779 20890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:33.290112 20890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:33.295473 20898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.295495 20896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.295490 20895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:33.295749 20890 server_base.cc:1034] running on GCE node
I20260925 12:25:33.296033 20890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:33.296344 20890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:33.297511 20890 hybrid_clock.cc:648] HybridClock initialized: now 1790339133297501 us; error 46 us; skew 500 ppm
I20260925 12:25:33.298882 20890 webserver.cc:460] Webserver started at http://127.1.47.190:39405/ using document root <none> and password file <none>
I20260925 12:25:33.299137 20890 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:33.299213 20890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:33.299371 20890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:33.300520 20890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "076a385d7c8e44a4b438a2b0626bc344"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b6a7321a63a15b41ce2f6cb836691522"
server_key_iv: "0e1aad4781064a0206d4452c0f724ec9"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.300892 20890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "076a385d7c8e44a4b438a2b0626bc344"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b6a7321a63a15b41ce2f6cb836691522"
server_key_iv: "0e1aad4781064a0206d4452c0f724ec9"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.302211 20890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:33.302958 20904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.303117 20890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:33.303217 20890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "076a385d7c8e44a4b438a2b0626bc344"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b6a7321a63a15b41ce2f6cb836691522"
server_key_iv: "0e1aad4781064a0206d4452c0f724ec9"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.303298 20890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:33.325846 20890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:33.326206 20890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:33.326368 20890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:33.330428 20890 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:42715
I20260925 12:25:33.330468 20956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:42715 every 8 connection(s)
I20260925 12:25:33.330843 20890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:33.331815 20957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:33.334105 20957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: Bootstrap starting.
I20260925 12:25:33.334698 20957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:33.335216 20957 log.cc:824] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:33.336010 20957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: No bootstrap required, opened a new log
I20260925 12:25:33.337793 20957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } }
I20260925 12:25:33.337942 20957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:33.337966 20957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076a385d7c8e44a4b438a2b0626bc344, State: Initialized, Role: FOLLOWER
I20260925 12:25:33.338169 20957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } }
I20260925 12:25:33.338261 20957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:33.338299 20957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:33.338347 20957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.338922 20957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } }
I20260925 12:25:33.339047 20957 leader_election.cc:302] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 076a385d7c8e44a4b438a2b0626bc344; no voters:
I20260925 12:25:33.339267 20957 leader_election.cc:288] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:33.339346 20960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:33.339517 20960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [term 1 LEADER]: Becoming Leader. State: Replica: 076a385d7c8e44a4b438a2b0626bc344, State: Running, Role: LEADER
I20260925 12:25:33.339576 20957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:33.339632 20960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } }
I20260925 12:25:33.339740 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20890
I20260925 12:25:33.339847 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:33.340117 1214 external_mini_cluster.cc:1428] Setting key 9c8d1830498b716be40546921c433f08
I20260925 12:25:33.340384 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 076a385d7c8e44a4b438a2b0626bc344. Latest consensus state: current_term: 1 leader_uuid: "076a385d7c8e44a4b438a2b0626bc344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } } }
I20260925 12:25:33.340462 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:33.340622 20961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "076a385d7c8e44a4b438a2b0626bc344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076a385d7c8e44a4b438a2b0626bc344" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42715 } } }
I20260925 12:25:33.340688 20961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:33.340662 20969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:33.341295 20969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:33.342963 20969 catalog_manager.cc:1347] Generated new cluster ID: 0e6cb233388e4bf0855924c6ff841e1d
I20260925 12:25:33.343014 20969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:33.346343 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.340868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.1.47.190:42715)
0925 12:25:33.341052 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.341062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.341095 (+ 33us) server_negotiation.cc:407] Connection header received
0925 12:25:33.341211 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.341220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.341289 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.341658 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:33.341749 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.342967 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.344200 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.344790 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.344909 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.345647 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:33.345651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:33.345651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:33.345857 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:33.345896 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:33.345909 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:33.345970 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.346081 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.346151 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260925 12:25:33.353686 20969 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:33.354260 20969 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:33.359328 20969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: Generated new TSK 0
I20260925 12:25:33.359514 20969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:33.363703 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.129:0
--local_ip_for_outbound_sockets=127.1.47.129
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--builtin_ntp_servers=127.1.47.148:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:33.445945 20981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:33.446198 20981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:33.446236 20981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:33.447434 20981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:33.447501 20981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:33.447546 20981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.129
I20260925 12:25:33.448966 20981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.47.129
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.129
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:33.449237 20981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:33.449546 20981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:33.455480 20987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.455492 20986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.455626 20989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:33.456072 20981 server_base.cc:1034] running on GCE node
I20260925 12:25:33.456311 20981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:33.456612 20981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:33.457819 20981 hybrid_clock.cc:648] HybridClock initialized: now 1790339133457797 us; error 34 us; skew 500 ppm
I20260925 12:25:33.459479 20981 webserver.cc:460] Webserver started at http://127.1.47.129:36385/ using document root <none> and password file <none>
I20260925 12:25:33.459928 20981 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:33.460007 20981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:33.460142 20981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:33.461638 20981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data/instance:
uuid: "71b676f24899485991652d9c96b25578"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "006d71f0fd879146e2c5ca9cfc8b00b5"
server_key_iv: "0ff4f00fa610ca0c97e54380d5ef1748"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.462110 20981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance:
uuid: "71b676f24899485991652d9c96b25578"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "006d71f0fd879146e2c5ca9cfc8b00b5"
server_key_iv: "0ff4f00fa610ca0c97e54380d5ef1748"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.463893 20981 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 12:25:33.464828 20995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.465019 20981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:33.465106 20981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal
uuid: "71b676f24899485991652d9c96b25578"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "006d71f0fd879146e2c5ca9cfc8b00b5"
server_key_iv: "0ff4f00fa610ca0c97e54380d5ef1748"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.465176 20981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:33.495460 20981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:33.495752 20981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:33.495872 20981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:33.496073 20981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:33.496443 20981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:33.496477 20981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.496539 20981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:33.496564 20981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.502507 20981 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.129:43755
I20260925 12:25:33.502555 21108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.129:43755 every 8 connection(s)
I20260925 12:25:33.502895 20981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/data/info.pb
I20260925 12:25:33.508514 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.503660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:34281 (local address 127.1.47.190:42715)
0925 12:25:33.503728 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.503730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.504030 (+ 300us) server_negotiation.cc:407] Connection header received
0925 12:25:33.504569 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.504571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.504585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.504630 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:33.505351 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.505904 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.507001 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.507202 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.507301 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.508220 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:33.508221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:33.508221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:33.508226 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:33.508242 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:33.508253 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:33.508317 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.508400 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.508433 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 12:25:33.508734 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.503659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42715 (local address 127.1.47.129:34281)
0925 12:25:33.503899 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.503942 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.504463 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.504720 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.504724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.504739 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:33.505248 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.505252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.505998 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.505999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.506897 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.506899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.507318 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.507319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.507465 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.508059 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:33.508059 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:33.508090 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:33.508112 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:33.508317 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:33.508323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:33.508327 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.508403 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.508508 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260925 12:25:33.509017 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 20981
I20260925 12:25:33.509135 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-0/wal/instance
I20260925 12:25:33.509291 21109 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42715
I20260925 12:25:33.509366 1214 external_mini_cluster.cc:1428] Setting key 2a475bdad7adbb6cc8efe0b6d6a12a9f
I20260925 12:25:33.509379 21109 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:33.509582 21109 heartbeater.cc:511] Master 127.1.47.190:42715 requested a full tablet report, sending...
I20260925 12:25:33.510141 20921 ts_manager.cc:194] Registered new tserver with Master: 71b676f24899485991652d9c96b25578 (127.1.47.129:43755)
I20260925 12:25:33.510387 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.130:0
--local_ip_for_outbound_sockets=127.1.47.130
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--builtin_ntp_servers=127.1.47.148:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 12:25:33.511057 20921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.129:34281
W20260925 12:25:33.591532 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:33.591763 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:33.591786 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:33.593019 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:33.593067 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:33.593094 21112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.130
I20260925 12:25:33.594399 21112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.47.130
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.130
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:33.594599 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:33.594880 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 12:25:33.600104 21112 server_base.cc:1034] running on GCE node
W20260925 12:25:33.600072 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.600072 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.600056 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:33.600663 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:33.600886 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:33.602026 21112 hybrid_clock.cc:648] HybridClock initialized: now 1790339133602014 us; error 30 us; skew 500 ppm
I20260925 12:25:33.603344 21112 webserver.cc:460] Webserver started at http://127.1.47.130:33267/ using document root <none> and password file <none>
I20260925 12:25:33.603600 21112 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:33.603649 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:33.603804 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:33.604928 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data/instance:
uuid: "e1bad48320a8499c94625302e944017d"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "78c38680f63801d0310dc26a80b118ac"
server_key_iv: "0ddd326651493d95d252d858ed0c4c72"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.605286 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance:
uuid: "e1bad48320a8499c94625302e944017d"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "78c38680f63801d0310dc26a80b118ac"
server_key_iv: "0ddd326651493d95d252d858ed0c4c72"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.606621 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:33.607326 21126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.607498 21112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:33.607602 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal
uuid: "e1bad48320a8499c94625302e944017d"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "78c38680f63801d0310dc26a80b118ac"
server_key_iv: "0ddd326651493d95d252d858ed0c4c72"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.607677 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:33.621315 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:33.621659 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:33.621819 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:33.622062 21112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:33.622398 21112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:33.622454 21112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.622496 21112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:33.622524 21112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.628616 21112 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.130:34761
I20260925 12:25:33.628758 21239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.130:34761 every 8 connection(s)
I20260925 12:25:33.629081 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/data/info.pb
I20260925 12:25:33.634209 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.629918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.130:52349 (local address 127.1.47.190:42715)
0925 12:25:33.629991 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.629993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.630203 (+ 210us) server_negotiation.cc:407] Connection header received
0925 12:25:33.630683 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.630684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.630696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.630789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:33.631456 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.631972 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.632912 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.633099 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.633157 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.634011 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:33.634012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:33.634013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:33.634016 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:33.634027 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:33.634030 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:33.634067 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.634127 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.634148 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 12:25:33.634451 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.629872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42715 (local address 127.1.47.130:52349)
0925 12:25:33.630046 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.630091 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.630574 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.630847 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.630850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.630858 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:33.631358 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.631363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.632051 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.632052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.632834 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.632836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.633173 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.633174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.633279 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.633878 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:33.633878 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:33.633907 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:33.633931 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:33.634069 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:33.634071 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:33.634073 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.634119 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.634197 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260925 12:25:33.634982 21240 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42715
I20260925 12:25:33.635090 21240 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:33.635273 21240 heartbeater.cc:511] Master 127.1.47.190:42715 requested a full tablet report, sending...
I20260925 12:25:33.635572 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21112
I20260925 12:25:33.635674 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-1/wal/instance
I20260925 12:25:33.635725 20921 ts_manager.cc:194] Registered new tserver with Master: e1bad48320a8499c94625302e944017d (127.1.47.130:34761)
I20260925 12:25:33.635905 1214 external_mini_cluster.cc:1428] Setting key 52e9acaadc122bfa1b27e840aa9b3286
I20260925 12:25:33.636269 20921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.130:52349
I20260925 12:25:33.637118 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.47.131:0
--local_ip_for_outbound_sockets=127.1.47.131
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--builtin_ntp_servers=127.1.47.148:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 12:25:33.722867 21243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:33.723068 21243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:33.723091 21243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:33.724335 21243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:33.724375 21243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:33.724421 21243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.47.131
I20260925 12:25:33.725764 21243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.47.131
--webserver_port=0
--tserver_master_addrs=127.1.47.190:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.47.131
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:33.726020 21243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:33.726327 21243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:33.731529 21249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:33.731529 21251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:33.731738 21243 server_base.cc:1034] running on GCE node
W20260925 12:25:33.731796 21248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:33.732056 21243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:33.732323 21243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:33.733491 21243 hybrid_clock.cc:648] HybridClock initialized: now 1790339133733470 us; error 36 us; skew 500 ppm
I20260925 12:25:33.734894 21243 webserver.cc:460] Webserver started at http://127.1.47.131:44833/ using document root <none> and password file <none>
I20260925 12:25:33.735189 21243 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:33.735282 21243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:33.735448 21243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:33.736609 21243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data/instance:
uuid: "84d23ff60d694785a4096f2b39028c35"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b313b58069e6a389ca95631adbf9c854"
server_key_iv: "4cc4fd31505b6995eb477046cbd1dc3d"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.737001 21243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance:
uuid: "84d23ff60d694785a4096f2b39028c35"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b313b58069e6a389ca95631adbf9c854"
server_key_iv: "4cc4fd31505b6995eb477046cbd1dc3d"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.738396 21243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:33.739284 21257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.739462 21243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:33.739568 21243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal
uuid: "84d23ff60d694785a4096f2b39028c35"
format_stamp: "Formatted at 2026-09-25 12:25:33 on dist-test-slave-l9ww"
server_key: "b313b58069e6a389ca95631adbf9c854"
server_key_iv: "4cc4fd31505b6995eb477046cbd1dc3d"
server_key_version: "encryptionkey@0"
I20260925 12:25:33.739643 21243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:33.771613 21243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:33.772039 21243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:33.772217 21243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:33.772465 21243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 12:25:33.772820 21243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 12:25:33.772884 21243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.772926 21243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 12:25:33.772954 21243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:33.778702 21243 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.131:33561
I20260925 12:25:33.778785 21370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.131:33561 every 8 connection(s)
I20260925 12:25:33.779121 21243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/data/info.pb
I20260925 12:25:33.782516 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21243
I20260925 12:25:33.782615 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal/instance
I20260925 12:25:33.782824 1214 external_mini_cluster.cc:1428] Setting key 99399faa43cc89a3e0bf4930f1d3e27e
I20260925 12:25:33.785099 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.780019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.131:45687 (local address 127.1.47.190:42715)
0925 12:25:33.780097 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.780099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.780298 (+ 199us) server_negotiation.cc:407] Connection header received
0925 12:25:33.780849 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.780850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.780860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.780917 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:33.781700 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.782182 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.783586 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.783799 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.783907 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.784784 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:33.784786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:33.784786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:33.784792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:33.784807 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:33.784814 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:33.784882 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.784986 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.785019 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 12:25:33.785377 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.779934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42715 (local address 127.1.47.131:45687)
0925 12:25:33.780154 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.780200 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.780761 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.781056 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.781060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.781069 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:33.781615 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.781620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.782354 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.782355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.783487 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.783490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.783908 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.783909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.784051 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.784602 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 12:25:33.784602 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:33.784633 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 12:25:33.784656 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:33.784882 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:33.784889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:33.784893 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.784989 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.785094 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260925 12:25:33.785856 21371 heartbeater.cc:347] Connected to a master server at 127.1.47.190:42715
I20260925 12:25:33.785950 21371 heartbeater.cc:464] Registering TS with master...
I20260925 12:25:33.786096 21371 heartbeater.cc:511] Master 127.1.47.190:42715 requested a full tablet report, sending...
I20260925 12:25:33.786481 20920 ts_manager.cc:194] Registered new tserver with Master: 84d23ff60d694785a4096f2b39028c35 (127.1.47.131:33561)
I20260925 12:25:33.786867 20920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.47.131:45687
I20260925 12:25:33.794629 1214 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 12:25:33.799243 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.796203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.1.47.190:42715)
0925 12:25:33.796279 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.796281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.796320 (+ 39us) server_negotiation.cc:407] Connection header received
0925 12:25:33.796402 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.796403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.796415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.796461 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:33.796627 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.797116 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.797961 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.798171 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.798245 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.798986 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:33.798987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:33.798988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:33.798992 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:33.799007 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:33.799011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:33.799065 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.799128 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.799155 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 12:25:33.801056 20920 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 12:25:33.802631 20920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 12:25:33.809083 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.804814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43755 (local address 127.0.0.1:47752)
0925 12:25:33.805020 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.805060 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.805126 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.805326 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.805329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.805332 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.805414 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.805416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.806598 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.806599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.807803 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.807805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.808771 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.808772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.808851 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.808869 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.808942 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.809001 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260925 12:25:33.809098 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.804931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.1.47.129:43755)
0925 12:25:33.805125 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.805130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.805142 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:25:33.805216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.805218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.805234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.805322 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.805476 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.806522 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.807937 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.808681 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.808759 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.808852 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.808972 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.809009 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260925 12:25:33.809540 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.805116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.1.47.131:33561)
0925 12:25:33.805558 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.805562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.805590 (+ 28us) server_negotiation.cc:407] Connection header received
0925 12:25:33.805628 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.805630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.805645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.805717 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.805859 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.806965 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.808188 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.808819 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.808896 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.808982 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.809398 (+ 416us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.809434 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":83,"threads_started":1}
I20260925 12:25:33.809805 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.804857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:33561 (local address 127.0.0.1:54586)
0925 12:25:33.805242 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.805262 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.805301 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.805715 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.805716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.805718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.805774 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.805776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.807056 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.807057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.808100 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.808103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.809014 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.809015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.809071 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.809074 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.809121 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.809162 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":52,"threads_started":1}
I20260925 12:25:33.809854 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.805277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.1.47.130:34761)
0925 12:25:33.805846 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.805851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.805861 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:33.805899 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.805901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.805915 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.805990 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.806114 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.807162 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.808349 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.809088 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.809507 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.809675 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.809701 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.809770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":89,"threads_started":1}
I20260925 12:25:33.809942 21043 tablet_service.cc:1467] Processing CreateTablet for tablet 869f88ed85474682936b58b9de1f71a2 (DEFAULT_TABLE table=test-table [id=ef51c04878d24eedb84e2ecdeed2dd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:33.810194 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.805057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:34761 (local address 127.0.0.1:60358)
0925 12:25:33.805392 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.805401 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.805436 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.806002 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.806003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.806005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.806052 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.806053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.807255 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.807256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.808265 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.808268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.809512 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.809513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.809572 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.809576 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.809624 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.809672 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":86,"threads_started":1}
I20260925 12:25:33.810374 21043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869f88ed85474682936b58b9de1f71a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:33.810626 21305 tablet_service.cc:1467] Processing CreateTablet for tablet 869f88ed85474682936b58b9de1f71a2 (DEFAULT_TABLE table=test-table [id=ef51c04878d24eedb84e2ecdeed2dd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:33.811002 21305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869f88ed85474682936b58b9de1f71a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:33.810943 21174 tablet_service.cc:1467] Processing CreateTablet for tablet 869f88ed85474682936b58b9de1f71a2 (DEFAULT_TABLE table=test-table [id=ef51c04878d24eedb84e2ecdeed2dd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 12:25:33.811299 21174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869f88ed85474682936b58b9de1f71a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:33.813519 21385 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: Bootstrap starting.
I20260925 12:25:33.814079 21385 tablet_bootstrap.cc:654] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:33.814373 21386 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Bootstrap starting.
I20260925 12:25:33.814440 21385 log.cc:824] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:33.814883 21388 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Bootstrap starting.
I20260925 12:25:33.815220 21385 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: No bootstrap required, opened a new log
I20260925 12:25:33.815230 21386 tablet_bootstrap.cc:654] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:33.815325 21385 ts_tablet_manager.cc:1397] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:25:33.815521 21388 tablet_bootstrap.cc:654] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:33.815562 21386 log.cc:824] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:33.815847 21388 log.cc:824] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:33.816454 21386 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: No bootstrap required, opened a new log
I20260925 12:25:33.816530 21386 ts_tablet_manager.cc:1397] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 12:25:33.818113 21385 raft_consensus.cc:348] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.818322 21385 raft_consensus.cc:374] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:33.818362 21385 raft_consensus.cc:729] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b676f24899485991652d9c96b25578, State: Initialized, Role: FOLLOWER
I20260925 12:25:33.818490 21385 consensus_queue.cc:260] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.818763 21385 ts_tablet_manager.cc:1428] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 12:25:33.818806 21109 heartbeater.cc:503] Master 127.1.47.190:42715 was elected leader, sending a full tablet report...
I20260925 12:25:33.819134 21386 raft_consensus.cc:348] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.819330 21386 raft_consensus.cc:374] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:33.819365 21386 raft_consensus.cc:729] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d23ff60d694785a4096f2b39028c35, State: Initialized, Role: FOLLOWER
I20260925 12:25:33.819475 21388 tablet_bootstrap.cc:492] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: No bootstrap required, opened a new log
I20260925 12:25:33.819505 21386 consensus_queue.cc:260] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.819578 21388 ts_tablet_manager.cc:1397] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 12:25:33.819803 21386 ts_tablet_manager.cc:1428] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 12:25:33.819841 21371 heartbeater.cc:503] Master 127.1.47.190:42715 was elected leader, sending a full tablet report...
I20260925 12:25:33.821899 21388 raft_consensus.cc:348] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.822067 21388 raft_consensus.cc:374] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:33.822113 21388 raft_consensus.cc:729] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1bad48320a8499c94625302e944017d, State: Initialized, Role: FOLLOWER
I20260925 12:25:33.822223 21388 consensus_queue.cc:260] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.822487 21388 ts_tablet_manager.cc:1428] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 12:25:33.822528 21240 heartbeater.cc:503] Master 127.1.47.190:42715 was elected leader, sending a full tablet report...
W20260925 12:25:33.879658 21241 tablet.cc:2369] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 12:25:33.907684 21391 raft_consensus.cc:482] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 12:25:33.907822 21391 raft_consensus.cc:504] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.908304 21391 leader_election.cc:288] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1bad48320a8499c94625302e944017d (127.1.47.130:34761), 84d23ff60d694785a4096f2b39028c35 (127.1.47.131:33561)
I20260925 12:25:33.912672 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.908712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:59369 (local address 127.1.47.131:33561)
0925 12:25:33.908756 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.908758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.908767 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:33.908825 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.908826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.908836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.908892 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.909159 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.909945 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.911675 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.912267 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.912359 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.912366 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.912554 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.912597 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:25:33.912672 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.908634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.47.129:41195 (local address 127.1.47.130:34761)
0925 12:25:33.908699 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.908701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.908713 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:25:33.908748 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.908749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.908763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.908851 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 12:25:33.909159 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.909931 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.911675 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.912267 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.912353 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.912363 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.912554 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.912597 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 12:25:33.912710 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.908529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:34761 (local address 127.1.47.129:41195)
0925 12:25:33.908603 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.908615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.908662 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.908892 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.908893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.908895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.909007 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.909009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.910049 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.910051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.911570 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.911573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.912380 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.912381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.912463 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.912468 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.912533 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.912593 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":101}
I20260925 12:25:33.912724 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.908529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.131:33561 (local address 127.1.47.129:59369)
0925 12:25:33.908712 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.908721 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:33.908775 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:33.908903 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:33.908904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:33.908905 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 12:25:33.909007 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.909009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.910049 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.910051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.911570 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:33.911573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.912380 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.912381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:33.912471 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.912473 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 12:25:33.912533 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 12:25:33.912592 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":167,"thread_start_us":106,"threads_started":1}
I20260925 12:25:33.913141 21194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869f88ed85474682936b58b9de1f71a2" candidate_uuid: "71b676f24899485991652d9c96b25578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bad48320a8499c94625302e944017d" is_pre_election: true
I20260925 12:25:33.913244 21325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869f88ed85474682936b58b9de1f71a2" candidate_uuid: "71b676f24899485991652d9c96b25578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d23ff60d694785a4096f2b39028c35" is_pre_election: true
I20260925 12:25:33.913316 21194 raft_consensus.cc:2454] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b676f24899485991652d9c96b25578 in term 0.
I20260925 12:25:33.913431 21325 raft_consensus.cc:2454] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b676f24899485991652d9c96b25578 in term 0.
I20260925 12:25:33.913590 20996 leader_election.cc:302] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b676f24899485991652d9c96b25578, e1bad48320a8499c94625302e944017d; no voters:
I20260925 12:25:33.913738 21391 raft_consensus.cc:2781] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 12:25:33.913792 21391 raft_consensus.cc:482] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 12:25:33.913810 21391 raft_consensus.cc:3037] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.914528 21391 raft_consensus.cc:504] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.914688 21391 leader_election.cc:288] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [CANDIDATE]: Term 1 election: Requested vote from peers e1bad48320a8499c94625302e944017d (127.1.47.130:34761), 84d23ff60d694785a4096f2b39028c35 (127.1.47.131:33561)
I20260925 12:25:33.914892 21325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869f88ed85474682936b58b9de1f71a2" candidate_uuid: "71b676f24899485991652d9c96b25578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d23ff60d694785a4096f2b39028c35"
I20260925 12:25:33.914909 21194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869f88ed85474682936b58b9de1f71a2" candidate_uuid: "71b676f24899485991652d9c96b25578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bad48320a8499c94625302e944017d"
I20260925 12:25:33.914978 21325 raft_consensus.cc:3037] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.914978 21194 raft_consensus.cc:3037] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:33.915958 21194 raft_consensus.cc:2454] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b676f24899485991652d9c96b25578 in term 1.
I20260925 12:25:33.915958 21325 raft_consensus.cc:2454] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b676f24899485991652d9c96b25578 in term 1.
I20260925 12:25:33.916178 20998 leader_election.cc:302] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b676f24899485991652d9c96b25578, 84d23ff60d694785a4096f2b39028c35; no voters:
I20260925 12:25:33.916275 21391 raft_consensus.cc:2781] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:33.916456 21391 raft_consensus.cc:686] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [term 1 LEADER]: Becoming Leader. State: Replica: 71b676f24899485991652d9c96b25578, State: Running, Role: LEADER
I20260925 12:25:33.916579 21391 consensus_queue.cc:237] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } }
I20260925 12:25:33.917557 20920 catalog_manager.cc:5495] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71b676f24899485991652d9c96b25578 (127.1.47.129). New cstate: current_term: 1 leader_uuid: "71b676f24899485991652d9c96b25578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b676f24899485991652d9c96b25578" member_type: VOTER last_known_addr { host: "127.1.47.129" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 } health_report { overall_health: UNKNOWN } } }
I20260925 12:25:33.924532 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.922029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.1.47.129:43755)
0925 12:25:33.922095 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:33.922097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:33.922106 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:33.922145 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:33.922146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:33.922165 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:33.922245 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:25:33.922397 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.922872 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.923650 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:33.923903 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:33.923995 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:33.924172 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:25:33.924288 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:25:33.924382 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:33.924406 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 12:25:33.924428 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:25:33.926345 21325 raft_consensus.cc:1261] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 1 FOLLOWER]: Refusing update from remote peer 71b676f24899485991652d9c96b25578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:33.926362 21194 raft_consensus.cc:1261] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 1 FOLLOWER]: Refusing update from remote peer 71b676f24899485991652d9c96b25578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:33.926757 21391 consensus_queue.cc:1035] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1bad48320a8499c94625302e944017d" member_type: VOTER last_known_addr { host: "127.1.47.130" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:33.926915 21395 consensus_queue.cc:1035] T 869f88ed85474682936b58b9de1f71a2 P 71b676f24899485991652d9c96b25578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d23ff60d694785a4096f2b39028c35" member_type: VOTER last_known_addr { host: "127.1.47.131" port: 33561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:33.929702 21399 mvcc.cc:205] Tried to move back new op lower bound from 7333229092559486976 to 7333229092523081728. Current Snapshot: MvccSnapshot[applied={T|T < 7333229092559486976}]
I20260925 12:25:33.930299 21400 mvcc.cc:205] Tried to move back new op lower bound from 7333229092559486976 to 7333229092523081728. Current Snapshot: MvccSnapshot[applied={T|T < 7333229092559486976 or (T in {7333229092559486976})}]
I20260925 12:25:33.932036 20920 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:33.932132 20920 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 12:25:33.932921 20920 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 076a385d7c8e44a4b438a2b0626bc344: Sending DeleteTablet for 3 replicas of tablet 869f88ed85474682936b58b9de1f71a2
I20260925 12:25:33.933230 21043 tablet_service.cc:1514] Processing DeleteTablet for tablet 869f88ed85474682936b58b9de1f71a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='slave'} at 127.0.0.1:47752
I20260925 12:25:33.933368 21174 tablet_service.cc:1514] Processing DeleteTablet for tablet 869f88ed85474682936b58b9de1f71a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='slave'} at 127.0.0.1:60358
I20260925 12:25:33.933435 21305 tablet_service.cc:1514] Processing DeleteTablet for tablet 869f88ed85474682936b58b9de1f71a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 12:25:33 UTC) from {username='slave'} at 127.0.0.1:54586
I20260925 12:25:33.933637 21412 tablet_replica.cc:330] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: stopping tablet replica
I20260925 12:25:33.933758 21412 raft_consensus.cc:2229] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 12:25:33.933861 21412 raft_consensus.cc:2258] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:33.933907 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20981
I20260925 12:25:33.934316 21412 ts_tablet_manager.cc:1905] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 12:25:33.936657 21412 ts_tablet_manager.cc:1918] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 12:25:33.936753 21412 log.cc:1196] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Deleting WAL directory at /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790338888974512-1214-0/minicluster-data/ts-2/wal/wals/869f88ed85474682936b58b9de1f71a2
I20260925 12:25:33.937013 21412 ts_tablet_manager.cc:1939] T 869f88ed85474682936b58b9de1f71a2 P 84d23ff60d694785a4096f2b39028c35: Deleting consensus metadata
I20260925 12:25:33.937757 20907 catalog_manager.cc:4841] TS 84d23ff60d694785a4096f2b39028c35 (127.1.47.131:33561): tablet 869f88ed85474682936b58b9de1f71a2 (table test-table [id=ef51c04878d24eedb84e2ecdeed2dd01]) successfully deleted
I20260925 12:25:33.939728 21411 tablet_replica.cc:330] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d: stopping tablet replica
I20260925 12:25:33.939906 21411 raft_consensus.cc:2229] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 12:25:33.939950 20905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.47.129:43755 (error 108)
I20260925 12:25:33.940053 21411 raft_consensus.cc:2258] T 869f88ed85474682936b58b9de1f71a2 P e1bad48320a8499c94625302e944017d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 12:25:33.940263 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21112
I20260925 12:25:33.941485 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.941347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.129:43755 (local address 127.0.0.1:47776)
0925 12:25:33.941401 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.941442 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:25:33.941603 20905 catalog_manager.cc:4584] TS 71b676f24899485991652d9c96b25578 (127.1.47.129:43755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 869f88ed85474682936b58b9de1f71a2: Network error: Client connection negotiation failed: client connection to 127.1.47.129:43755: connect: Connection refused (error 111)
I20260925 12:25:33.944643 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21243
I20260925 12:25:33.944692 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:33.944549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.130:34761 (local address 127.0.0.1:60368)
0925 12:25:33.944635 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:33.944663 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.130:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 12:25:33.949175 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 20890
2026-09-25T12:25:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (770 ms)
[----------] 6 tests from EncryptionPolicyTest (8665 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:34 dist-test-slave-l9ww krb5kdc[21416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:34 dist-test-slave-l9ww krb5kdc[21416](info): set up 2 sockets
Sep 25 12:25:34 dist-test-slave-l9ww krb5kdc[21416](info): commencing operation
krb5kdc: starting...
W20260925 12:25:34.984205 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:35.106710 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:35.189666 21429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:35.189898 21429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:35.189922 21429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:35.191068 21429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:35.191107 21429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:35.191123 21429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:35.191134 21429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:35.191146 21429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:35.192773 21429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:35.193003 21429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:35.193192 21429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:35.198169 21437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:35.198268 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:35.198290 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:35.198563 21429 server_base.cc:1034] running on GCE node
I20260925 12:25:35.198833 21429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:35.199108 21429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:35.200289 21429 hybrid_clock.cc:648] HybridClock initialized: now 1790339135200269 us; error 34 us; skew 500 ppm
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339135, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:35.203085 21429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:35.203660 21429 webserver.cc:460] Webserver started at http://127.0.0.1:45987/ using document root <none> and password file <none>
I20260925 12:25:35.204005 21429 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:35.204079 21429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:35.204242 21429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:35.205379 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "2b7684388c7e4ffd8c7de97c66b62589"
format_stamp: "Formatted at 2026-09-25 12:25:35 on dist-test-slave-l9ww"
server_key: "1daa733e5316985845ccfc1b6dce906f"
server_key_iv: "d36b6fa989b41f1eca3c5f27b53a608d"
server_key_version: "encryptionkey@0"
I20260925 12:25:35.205772 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "2b7684388c7e4ffd8c7de97c66b62589"
format_stamp: "Formatted at 2026-09-25 12:25:35 on dist-test-slave-l9ww"
server_key: "1daa733e5316985845ccfc1b6dce906f"
server_key_iv: "d36b6fa989b41f1eca3c5f27b53a608d"
server_key_version: "encryptionkey@0"
I20260925 12:25:35.207247 21429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:35.208259 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:35.208442 21429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:35.208539 21429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "2b7684388c7e4ffd8c7de97c66b62589"
format_stamp: "Formatted at 2026-09-25 12:25:35 on dist-test-slave-l9ww"
server_key: "1daa733e5316985845ccfc1b6dce906f"
server_key_iv: "d36b6fa989b41f1eca3c5f27b53a608d"
server_key_version: "encryptionkey@0"
I20260925 12:25:35.208633 21429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:35.232244 21429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:35.234602 21429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:35.234786 21429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:35.239086 21429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33123
I20260925 12:25:35.239122 21496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33123 every 8 connection(s)
I20260925 12:25:35.239532 21429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:35.240602 21497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:35.242295 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21429
I20260925 12:25:35.242393 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:35.242654 1214 external_mini_cluster.cc:1428] Setting key 37805914793cb2726fe6d63147e4ba45
I20260925 12:25:35.243225 21497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589: Bootstrap starting.
I20260925 12:25:35.243932 21497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:35.244226 21497 log.cc:824] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:35.245132 21497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589: No bootstrap required, opened a new log
I20260925 12:25:35.247468 21497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } }
I20260925 12:25:35.247682 21497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:35.247712 21497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7684388c7e4ffd8c7de97c66b62589, State: Initialized, Role: FOLLOWER
I20260925 12:25:35.247893 21497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } }
I20260925 12:25:35.247977 21497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:35.248003 21497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:35.248070 21497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:35.248915 21497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } }
I20260925 12:25:35.249038 21497 leader_election.cc:302] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b7684388c7e4ffd8c7de97c66b62589; no voters:
I20260925 12:25:35.249223 21497 leader_election.cc:288] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:35.249253 21502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:35.249485 21502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [term 1 LEADER]: Becoming Leader. State: Replica: 2b7684388c7e4ffd8c7de97c66b62589, State: Running, Role: LEADER
I20260925 12:25:35.249516 21497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:35.249590 21502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } }
I20260925 12:25:35.249891 21502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b7684388c7e4ffd8c7de97c66b62589. Latest consensus state: current_term: 1 leader_uuid: "2b7684388c7e4ffd8c7de97c66b62589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } } }
I20260925 12:25:35.249902 21504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b7684388c7e4ffd8c7de97c66b62589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7684388c7e4ffd8c7de97c66b62589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33123 } } }
I20260925 12:25:35.249990 21504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:35.250186 21502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:35.250264 21506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 12:25:35.250765 21506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:35.252647 21506 catalog_manager.cc:1347] Generated new cluster ID: 458f82e25cd746f7bf0bfa474eb8fe3a
I20260925 12:25:35.252705 21506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:35.258986 21506 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:35.258992 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:35.243402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35165 (local address 127.0.0.1:33123)
0925 12:25:35.243616 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:35.243627 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:35.243656 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:25:35.244254 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:35.244262 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:35.244340 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:35.244548 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:35.245373 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:35.246296 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:35.247614 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:35.248283 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:35.248370 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:35.251387 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:35.251391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:35.251391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:35.251405 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:35.257011 (+ 5606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:35.257562 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:35.257566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:35.257567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:35.257717 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:35.257992 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:35.257993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:35.257993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:35.258323 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:35.258497 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:35.258661 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 12:25:35.258773 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260925 12:25:35.259985 21506 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:35.260135 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:35.266952 21506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b7684388c7e4ffd8c7de97c66b62589: Generated new TSK 0
I20260925 12:25:35.267170 21506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:35.269290 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:35.261680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36147 (local address 127.0.0.1:33123)
0925 12:25:35.261717 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:35.261719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:35.261733 (+ 14us) server_negotiation.cc:407] Connection header received
0925 12:25:35.261803 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:35.261803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:35.261814 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:35.261871 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:35.262450 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:35.263276 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:35.264324 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:35.264662 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:35.264736 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:35.266167 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:35.266168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:35.266168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:35.266176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:35.268007 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:35.268445 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:35.268446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:35.268446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:35.268515 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:35.268795 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:35.268796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:35.268797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:35.268929 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:35.269045 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:35.269136 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 12:25:35.269199 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:25:35.271345 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21429
W20260925 12:25:35.275400 21414 connection.cc:535] client connection to 127.0.0.1:33123 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33123): Connection reset by peer (error 104)
2026-09-25T12:25:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21528](info): set up 2 sockets
Sep 25 12:25:35 dist-test-slave-l9ww krb5kdc[21528](info): commencing operation
krb5kdc: starting...
W20260925 12:25:36.307806 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:36 dist-test-slave-l9ww krb5kdc[21528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:36.427232 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:36.510174 21541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:36.510406 21541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:36.510432 21541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:36.511682 21541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:36.511741 21541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:36.511756 21541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:36.511804 21541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:36.511833 21541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:36.513456 21541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39345
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:36.513747 21541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:36.513994 21541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:36.519107 21547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:36.519188 21541 server_base.cc:1034] running on GCE node
W20260925 12:25:36.519146 21546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:36.519171 21549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:36.519605 21541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:36.519970 21541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:36.521126 21541 hybrid_clock.cc:648] HybridClock initialized: now 1790339136521111 us; error 31 us; skew 500 ppm
Sep 25 12:25:36 dist-test-slave-l9ww krb5kdc[21528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:36.529542 21541 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:36.530007 21541 webserver.cc:460] Webserver started at http://127.0.0.1:41955/ using document root <none> and password file <none>
I20260925 12:25:36.530212 21541 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:36.530257 21541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:36.530349 21541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:36.531416 21541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "2e49a33a2b6947c9a623eec76e1d2efb"
format_stamp: "Formatted at 2026-09-25 12:25:36 on dist-test-slave-l9ww"
server_key: "eb31202e878169d05a9498acad63538f"
server_key_iv: "dd417ec5db2c294025bcd2dd5635c3cc"
server_key_version: "encryptionkey@0"
I20260925 12:25:36.531723 21541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "2e49a33a2b6947c9a623eec76e1d2efb"
format_stamp: "Formatted at 2026-09-25 12:25:36 on dist-test-slave-l9ww"
server_key: "eb31202e878169d05a9498acad63538f"
server_key_iv: "dd417ec5db2c294025bcd2dd5635c3cc"
server_key_version: "encryptionkey@0"
I20260925 12:25:36.533213 21541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:25:36.533936 21556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:36.534116 21541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:36.534191 21541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "2e49a33a2b6947c9a623eec76e1d2efb"
format_stamp: "Formatted at 2026-09-25 12:25:36 on dist-test-slave-l9ww"
server_key: "eb31202e878169d05a9498acad63538f"
server_key_iv: "dd417ec5db2c294025bcd2dd5635c3cc"
server_key_version: "encryptionkey@0"
I20260925 12:25:36.534248 21541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:36.551122 21541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:36.553655 21541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:36.553784 21541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:36.558173 21541 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39345
I20260925 12:25:36.558210 21608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39345 every 8 connection(s)
I20260925 12:25:36.558602 21541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:36.559604 21609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:36.562024 21609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb: Bootstrap starting.
I20260925 12:25:36.562615 21609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:36.562736 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21541
I20260925 12:25:36.562845 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:36.562983 21609 log.cc:824] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:36.563035 1214 external_mini_cluster.cc:1428] Setting key c11b0a04adab43fa70beb286874979a5
I20260925 12:25:36.564108 21609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb: No bootstrap required, opened a new log
I20260925 12:25:36.566393 21609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } }
I20260925 12:25:36.566568 21609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:36.566599 21609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e49a33a2b6947c9a623eec76e1d2efb, State: Initialized, Role: FOLLOWER
I20260925 12:25:36.566721 21609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } }
I20260925 12:25:36.566795 21609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:36.566843 21609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:36.566929 21609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:36.567610 21609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } }
I20260925 12:25:36.567735 21609 leader_election.cc:302] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e49a33a2b6947c9a623eec76e1d2efb; no voters:
I20260925 12:25:36.567997 21609 leader_election.cc:288] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:36.568032 21614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:36.568197 21614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [term 1 LEADER]: Becoming Leader. State: Replica: 2e49a33a2b6947c9a623eec76e1d2efb, State: Running, Role: LEADER
I20260925 12:25:36.568279 21609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:36.568290 21614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } }
I20260925 12:25:36.568615 21615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } } }
I20260925 12:25:36.568688 21615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:36.568861 21616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e49a33a2b6947c9a623eec76e1d2efb. Latest consensus state: current_term: 1 leader_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e49a33a2b6947c9a623eec76e1d2efb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39345 } } }
I20260925 12:25:36.568933 21616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:36.569252 21619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:36.569408 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:36.563592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35247 (local address 127.0.0.1:39345)
0925 12:25:36.563813 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:36.563821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:36.563850 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:25:36.564427 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:36.564435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:36.564496 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:36.564614 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:36.564838 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:36.565687 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:36.566858 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:36.567366 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:36.567447 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:36.568502 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:36.568506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:36.568506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:36.568713 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:36.568757 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:36.568772 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:36.568839 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:36.569184 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 12:25:36.569216 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260925 12:25:36.569718 21619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:36.570451 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:36.574697 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:36.571786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58559 (local address 127.0.0.1:39345)
0925 12:25:36.571823 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:36.571825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:36.571977 (+ 152us) server_negotiation.cc:407] Connection header received
0925 12:25:36.572037 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:36.572038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:36.572048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:36.572097 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:36.572240 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:36.572717 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:36.573447 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:36.573676 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:36.573760 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:36.574470 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:36.574472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:36.574472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:36.574477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:36.574492 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:36.574496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:36.574549 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:36.574609 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 12:25:36.574618 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:25:36.576913 21619 catalog_manager.cc:1347] Generated new cluster ID: 58b2f6499bad4580941b7380a193c4be
I20260925 12:25:36.576977 21619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:36.589223 21619 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:36.590217 21619 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:36.592442 21619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e49a33a2b6947c9a623eec76e1d2efb: Generated new TSK 0
I20260925 12:25:36.592641 21619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:36.601075 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21541
W20260925 12:25:36.605127 21526 connection.cc:535] client connection to 127.0.0.1:39345 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39345): Connection reset by peer (error 104)
2026-09-25T12:25:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:36 dist-test-slave-l9ww krb5kdc[21640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:36 dist-test-slave-l9ww krb5kdc[21640](info): set up 2 sockets
Sep 25 12:25:36 dist-test-slave-l9ww krb5kdc[21640](info): commencing operation
krb5kdc: starting...
W20260925 12:25:37.696194 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:37 dist-test-slave-l9ww krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:37.814877 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:37.896111 21653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:37.896334 21653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:37.896358 21653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:37.897476 21653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:37.897513 21653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:37.897527 21653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:37.897540 21653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:37.897552 21653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:37.898964 21653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:37.899207 21653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:37.899430 21653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:37.904393 21659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:37.904386 21661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:37.904386 21658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:37.904640 21653 server_base.cc:1034] running on GCE node
I20260925 12:25:37.904860 21653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:37.905097 21653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:37.906237 21653 hybrid_clock.cc:648] HybridClock initialized: now 1790339137906225 us; error 30 us; skew 500 ppm
Sep 25 12:25:37 dist-test-slave-l9ww krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339137, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:37.908836 21653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:37.909271 21653 webserver.cc:460] Webserver started at http://127.0.0.1:44497/ using document root <none> and password file <none>
I20260925 12:25:37.909504 21653 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:37.909574 21653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:37.909725 21653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:37.910794 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "1365ba1bb6654271a90086cb63da6b39"
format_stamp: "Formatted at 2026-09-25 12:25:37 on dist-test-slave-l9ww"
server_key: "4282a0209232615f8e196c6f8dcfe312"
server_key_iv: "7608d5a323260326ea58df980e938501"
server_key_version: "encryptionkey@0"
I20260925 12:25:37.911162 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "1365ba1bb6654271a90086cb63da6b39"
format_stamp: "Formatted at 2026-09-25 12:25:37 on dist-test-slave-l9ww"
server_key: "4282a0209232615f8e196c6f8dcfe312"
server_key_iv: "7608d5a323260326ea58df980e938501"
server_key_version: "encryptionkey@0"
I20260925 12:25:37.912557 21653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:25:37.913275 21668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:37.913444 21653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:25:37.913549 21653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "1365ba1bb6654271a90086cb63da6b39"
format_stamp: "Formatted at 2026-09-25 12:25:37 on dist-test-slave-l9ww"
server_key: "4282a0209232615f8e196c6f8dcfe312"
server_key_iv: "7608d5a323260326ea58df980e938501"
server_key_version: "encryptionkey@0"
I20260925 12:25:37.913625 21653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:37.917057 21653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:37.918938 21653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:37.919091 21653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:37.922917 21653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39307
I20260925 12:25:37.922946 21720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39307 every 8 connection(s)
I20260925 12:25:37.923273 21653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:37.924304 21721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:37.926712 21721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39: Bootstrap starting.
I20260925 12:25:37.927315 21721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:37.927603 21721 log.cc:824] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:37.928418 21721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39: No bootstrap required, opened a new log
I20260925 12:25:37.929854 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21653
I20260925 12:25:37.930001 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:37.930174 1214 external_mini_cluster.cc:1428] Setting key 68a88a0ab8184b75a4334645a7e5c938
I20260925 12:25:37.930087 21721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260925 12:25:37.930248 21721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:37.930294 21721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1365ba1bb6654271a90086cb63da6b39, State: Initialized, Role: FOLLOWER
I20260925 12:25:37.930444 21721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260925 12:25:37.930527 21721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:37.930572 21721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:37.930619 21721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:37.931735 21721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260925 12:25:37.931895 21721 leader_election.cc:302] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1365ba1bb6654271a90086cb63da6b39; no voters:
I20260925 12:25:37.932070 21721 leader_election.cc:288] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:37.932129 21726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:37.932430 21721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:37.932570 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:37.930761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35407 (local address 127.0.0.1:39307)
0925 12:25:37.930955 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:37.930965 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:37.931002 (+ 37us) server_negotiation.cc:407] Connection header received
0925 12:25:37.931107 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:37.931112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:37.931136 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:37.931244 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:37.931995 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:37.932000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:37.932000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:37.932005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:37.932055 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:37.932071 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:37.932139 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:37.932283 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 12:25:37.932328 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260925 12:25:37.932387 21726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [term 1 LEADER]: Becoming Leader. State: Replica: 1365ba1bb6654271a90086cb63da6b39, State: Running, Role: LEADER
I20260925 12:25:37.932928 21726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260925 12:25:37.933558 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:37.933619 21728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1365ba1bb6654271a90086cb63da6b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } } }
I20260925 12:25:37.933709 21728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:37.934110 21741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:37.934643 21741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:37.935045 21727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1365ba1bb6654271a90086cb63da6b39. Latest consensus state: current_term: 1 leader_uuid: "1365ba1bb6654271a90086cb63da6b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1365ba1bb6654271a90086cb63da6b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } } }
I20260925 12:25:37.935112 21727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:37.936400 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:37.935236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36017 (local address 127.0.0.1:39307)
0925 12:25:37.935275 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:37.935277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:37.935398 (+ 121us) server_negotiation.cc:407] Connection header received
0925 12:25:37.935428 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:37.935429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:37.935435 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:37.935503 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:37.936164 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:37.936165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:37.936166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:37.936170 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:37.936185 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:37.936189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:37.936251 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:37.936316 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 12:25:37.936330 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:25:37.936467 21741 catalog_manager.cc:1347] Generated new cluster ID: 1d507863405f4dd280d0acb6431b0c05
I20260925 12:25:37.936511 21741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:37.944803 21741 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:37.944988 21741 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:37.948954 21741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1365ba1bb6654271a90086cb63da6b39: Generated new TSK 0
I20260925 12:25:37.949163 21741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:37.954792 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21653
2026-09-25T12:25:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:38 dist-test-slave-l9ww krb5kdc[21752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:38 dist-test-slave-l9ww krb5kdc[21752](info): set up 2 sockets
Sep 25 12:25:38 dist-test-slave-l9ww krb5kdc[21752](info): commencing operation
krb5kdc: starting...
W20260925 12:25:38.992041 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:39.112743 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:39.195650 21765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:39.195919 21765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:39.195947 21765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:39.197224 21765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:39.197273 21765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:39.197288 21765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:39.197300 21765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:39.197312 21765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:39.198809 21765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:39.199026 21765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:39.199216 21765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:39.204126 21770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:39.204133 21771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:39.204133 21773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:39.204566 21765 server_base.cc:1034] running on GCE node
I20260925 12:25:39.204869 21765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:39.205183 21765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:39.206359 21765 hybrid_clock.cc:648] HybridClock initialized: now 1790339139206330 us; error 43 us; skew 500 ppm
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339139, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:39.209272 21765 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:39.209848 21765 webserver.cc:460] Webserver started at http://127.0.0.1:44535/ using document root <none> and password file <none>
I20260925 12:25:39.210117 21765 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:39.210191 21765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:39.210340 21765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:39.211522 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "59271302f9614095aa78a6f817e06101"
format_stamp: "Formatted at 2026-09-25 12:25:39 on dist-test-slave-l9ww"
server_key: "d63fda3f4a993c8af5076e513341e459"
server_key_iv: "a9535d4c47c46d04534fee2da70074a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:39.212086 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "59271302f9614095aa78a6f817e06101"
format_stamp: "Formatted at 2026-09-25 12:25:39 on dist-test-slave-l9ww"
server_key: "d63fda3f4a993c8af5076e513341e459"
server_key_iv: "a9535d4c47c46d04534fee2da70074a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:39.214062 21765 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 12:25:39.215006 21780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:39.215230 21765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:39.215339 21765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "59271302f9614095aa78a6f817e06101"
format_stamp: "Formatted at 2026-09-25 12:25:39 on dist-test-slave-l9ww"
server_key: "d63fda3f4a993c8af5076e513341e459"
server_key_iv: "a9535d4c47c46d04534fee2da70074a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:39.215435 21765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:39.230476 21765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:39.232985 21765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:39.233168 21765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:39.237421 21765 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42365
I20260925 12:25:39.237475 21832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42365 every 8 connection(s)
I20260925 12:25:39.237874 21765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:39.238893 21833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:39.241513 21833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101: Bootstrap starting.
I20260925 12:25:39.242158 21833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:39.242465 21833 log.cc:824] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:39.243273 21833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101: No bootstrap required, opened a new log
I20260925 12:25:39.244987 21833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } }
I20260925 12:25:39.245137 21833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:39.245205 21833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59271302f9614095aa78a6f817e06101, State: Initialized, Role: FOLLOWER
I20260925 12:25:39.245414 21833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } }
I20260925 12:25:39.245503 21833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:39.245558 21833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:39.245611 21833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:39.246194 21833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } }
I20260925 12:25:39.246315 21833 leader_election.cc:302] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59271302f9614095aa78a6f817e06101; no voters:
I20260925 12:25:39.246505 21833 leader_election.cc:288] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:39.246572 21836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:39.246774 21836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [term 1 LEADER]: Becoming Leader. State: Replica: 59271302f9614095aa78a6f817e06101, State: Running, Role: LEADER
I20260925 12:25:39.246805 21833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:39.246932 21836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } }
I20260925 12:25:39.247359 21837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59271302f9614095aa78a6f817e06101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } } }
I20260925 12:25:39.247354 21838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59271302f9614095aa78a6f817e06101. Latest consensus state: current_term: 1 leader_uuid: "59271302f9614095aa78a6f817e06101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59271302f9614095aa78a6f817e06101" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42365 } } }
I20260925 12:25:39.247454 21837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:39.247462 21838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:39.247661 21840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:39.248070 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21765
I20260925 12:25:39.248174 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:39.248281 21840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:39.248363 1214 external_mini_cluster.cc:1428] Setting key fc15f01560b316a0df2d447b196bce73
I20260925 12:25:39.250284 21840 catalog_manager.cc:1347] Generated new cluster ID: 10eab57efeb34f28b6ae408311cd2554
I20260925 12:25:39.250346 21840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 12:25:39.256420 21840 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:39.256989 21840 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:39.260794 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:39.248994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39365 (local address 127.0.0.1:42365)
0925 12:25:39.249168 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:39.249178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:39.249388 (+ 210us) server_negotiation.cc:407] Connection header received
0925 12:25:39.249554 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:39.249561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:39.249645 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:39.249793 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:39.250491 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:39.251950 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:39.253046 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:39.253700 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:39.253836 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:39.256996 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:39.257002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:39.257002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:39.257016 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:39.259074 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:39.259513 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:39.259516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:39.259517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:39.259625 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:39.259836 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:39.259837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:39.259837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:39.260137 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:39.260296 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:39.260412 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 12:25:39.260555 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260925 12:25:39.261809 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:39.261830 21840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59271302f9614095aa78a6f817e06101: Generated new TSK 0
I20260925 12:25:39.262006 21840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:39.270489 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:39.263390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53285 (local address 127.0.0.1:42365)
0925 12:25:39.263430 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:39.263432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:39.263567 (+ 135us) server_negotiation.cc:407] Connection header received
0925 12:25:39.263626 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:39.263627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:39.263647 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:39.263705 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:39.264274 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:39.265084 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:39.265877 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:39.266218 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:39.266287 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:39.267537 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:39.267539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:39.267540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:39.267548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:39.269405 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:39.269749 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:39.269750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:39.269750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:39.269821 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:39.269998 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:39.269999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:39.269999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:39.270132 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:39.270230 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:39.270307 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 12:25:39.270389 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:25:39.272631 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21765
2026-09-25T12:25:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21864](info): set up 2 sockets
Sep 25 12:25:39 dist-test-slave-l9ww krb5kdc[21864](info): commencing operation
krb5kdc: starting...
W20260925 12:25:40.328084 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:40 dist-test-slave-l9ww krb5kdc[21864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:40.451318 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:40.537247 21877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:40.537508 21877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:40.537544 21877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:40.538709 21877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:40.538775 21877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:40.538800 21877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:40.538829 21877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:40.538858 21877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:40.540385 21877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:40.540674 21877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:40.540901 21877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:40.545943 21885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:40.546069 21877 server_base.cc:1034] running on GCE node
W20260925 12:25:40.546154 21882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:40.545959 21883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:40.546432 21877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:40.546734 21877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:40.547940 21877 hybrid_clock.cc:648] HybridClock initialized: now 1790339140547926 us; error 29 us; skew 500 ppm
Sep 25 12:25:40 dist-test-slave-l9ww krb5kdc[21864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:40.550446 21877 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:40.550935 21877 webserver.cc:460] Webserver started at http://127.0.0.1:37323/ using document root <none> and password file <none>
I20260925 12:25:40.551177 21877 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:40.551242 21877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:40.551393 21877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:40.552553 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "9f6a19114b484414b574471860a83a6f"
format_stamp: "Formatted at 2026-09-25 12:25:40 on dist-test-slave-l9ww"
server_key: "f18b9c9b6c0865828a6824cd529d3fd6"
server_key_iv: "2b8d53982753858273adac6b5eb0eed4"
server_key_version: "encryptionkey@0"
I20260925 12:25:40.552927 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "9f6a19114b484414b574471860a83a6f"
format_stamp: "Formatted at 2026-09-25 12:25:40 on dist-test-slave-l9ww"
server_key: "f18b9c9b6c0865828a6824cd529d3fd6"
server_key_iv: "2b8d53982753858273adac6b5eb0eed4"
server_key_version: "encryptionkey@0"
I20260925 12:25:40.554255 21877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:40.554999 21892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:40.555172 21877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:40.555279 21877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "9f6a19114b484414b574471860a83a6f"
format_stamp: "Formatted at 2026-09-25 12:25:40 on dist-test-slave-l9ww"
server_key: "f18b9c9b6c0865828a6824cd529d3fd6"
server_key_iv: "2b8d53982753858273adac6b5eb0eed4"
server_key_version: "encryptionkey@0"
I20260925 12:25:40.555354 21877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:40.582237 21877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:40.584326 21877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:40.584501 21877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:40.588822 21877 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44865
I20260925 12:25:40.588888 21944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44865 every 8 connection(s)
I20260925 12:25:40.589260 21877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:40.590232 21945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:40.592694 21945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f: Bootstrap starting.
I20260925 12:25:40.593305 21945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:40.593597 21945 log.cc:824] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:40.594352 21945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f: No bootstrap required, opened a new log
I20260925 12:25:40.596150 21945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260925 12:25:40.596300 21945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:40.596324 21945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6a19114b484414b574471860a83a6f, State: Initialized, Role: FOLLOWER
I20260925 12:25:40.596477 21945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260925 12:25:40.596581 21945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:40.596619 21945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:40.596666 21945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:40.596696 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21877
I20260925 12:25:40.596808 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:40.596982 1214 external_mini_cluster.cc:1428] Setting key dba1b6b146224fa8a0420ee778b715fc
I20260925 12:25:40.597394 21945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260925 12:25:40.597530 21945 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f6a19114b484414b574471860a83a6f; no voters:
I20260925 12:25:40.597715 21945 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:40.597746 21950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:40.597879 21950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [term 1 LEADER]: Becoming Leader. State: Replica: 9f6a19114b484414b574471860a83a6f, State: Running, Role: LEADER
I20260925 12:25:40.597994 21945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:40.597965 21950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260925 12:25:40.598392 21952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f6a19114b484414b574471860a83a6f. Latest consensus state: current_term: 1 leader_uuid: "9f6a19114b484414b574471860a83a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } } }
I20260925 12:25:40.598487 21952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:40.598469 21951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f6a19114b484414b574471860a83a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6a19114b484414b574471860a83a6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } } }
I20260925 12:25:40.598549 21951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:40.599165 21954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:40.599670 21954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:40.601392 21954 catalog_manager.cc:1347] Generated new cluster ID: 08cca4be5d344a64a645c81e32754338
I20260925 12:25:40.601449 21954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:40.603704 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:40.597516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45545 (local address 127.0.0.1:44865)
0925 12:25:40.597742 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:40.597752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:40.597808 (+ 56us) server_negotiation.cc:407] Connection header received
0925 12:25:40.598125 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:40.598133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:40.598195 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:40.598359 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:40.598718 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:40.600135 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:40.601240 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:40.601849 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:40.601950 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:40.602802 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:40.602806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:40.602807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:40.603041 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:40.603079 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:40.603093 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:40.603151 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:40.603411 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 12:25:40.603474 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260925 12:25:40.604528 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:40.609766 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:40.606077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54367 (local address 127.0.0.1:44865)
0925 12:25:40.606113 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:40.606115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:40.606229 (+ 114us) server_negotiation.cc:407] Connection header received
0925 12:25:40.606327 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:40.606328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:40.606339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:40.606369 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:40.606565 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:40.607286 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:40.608323 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:40.608624 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:40.608665 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:40.609478 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:40.609479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:40.609480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:40.609485 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:40.609499 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:40.609503 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:40.609532 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:40.609652 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 12:25:40.609679 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:25:40.618443 21954 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:40.619030 21954 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:40.622130 21954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f6a19114b484414b574471860a83a6f: Generated new TSK 0
I20260925 12:25:40.622351 21954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:40.624732 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21877
2026-09-25T12:25:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:40 dist-test-slave-l9ww krb5kdc[21976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:40 dist-test-slave-l9ww krb5kdc[21976](info): set up 2 sockets
Sep 25 12:25:40 dist-test-slave-l9ww krb5kdc[21976](info): commencing operation
krb5kdc: starting...
W20260925 12:25:41.672173 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:41 dist-test-slave-l9ww krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:41.793087 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:41.874203 21989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:41.874418 21989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:41.874441 21989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:41.875582 21989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:41.875622 21989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:41.875635 21989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:41.875648 21989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:41.875660 21989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:41.877101 21989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:41.877406 21989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:41.877669 21989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:41.882532 21997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:41.882541 21995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:41.882532 21994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:41.883324 21989 server_base.cc:1034] running on GCE node
I20260925 12:25:41.883574 21989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:41.883924 21989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:41.885071 21989 hybrid_clock.cc:648] HybridClock initialized: now 1790339141885054 us; error 39 us; skew 500 ppm
Sep 25 12:25:41 dist-test-slave-l9ww krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:41.893620 21989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:41.894141 21989 webserver.cc:460] Webserver started at http://127.0.0.1:38989/ using document root <none> and password file <none>
I20260925 12:25:41.894402 21989 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:41.894474 21989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:41.894623 21989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:41.895725 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "eef2ebcb273c4ace93f47c79902f1331"
format_stamp: "Formatted at 2026-09-25 12:25:41 on dist-test-slave-l9ww"
server_key: "d348693097c84c8dda5a9bd554f58334"
server_key_iv: "b90c05b425166c465c54fdf233dbf442"
server_key_version: "encryptionkey@0"
I20260925 12:25:41.896160 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "eef2ebcb273c4ace93f47c79902f1331"
format_stamp: "Formatted at 2026-09-25 12:25:41 on dist-test-slave-l9ww"
server_key: "d348693097c84c8dda5a9bd554f58334"
server_key_iv: "b90c05b425166c465c54fdf233dbf442"
server_key_version: "encryptionkey@0"
I20260925 12:25:41.897651 21989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:41.898491 22004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:41.898684 21989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:41.898762 21989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "eef2ebcb273c4ace93f47c79902f1331"
format_stamp: "Formatted at 2026-09-25 12:25:41 on dist-test-slave-l9ww"
server_key: "d348693097c84c8dda5a9bd554f58334"
server_key_iv: "b90c05b425166c465c54fdf233dbf442"
server_key_version: "encryptionkey@0"
I20260925 12:25:41.898828 21989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:41.902053 21989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:41.904019 21989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:41.904129 21989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:41.907892 21989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43935
I20260925 12:25:41.907950 22056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43935 every 8 connection(s)
I20260925 12:25:41.908277 21989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:41.909261 22057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:41.911442 22057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331: Bootstrap starting.
I20260925 12:25:41.912091 22057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:41.912376 22057 log.cc:824] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:41.913035 22057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331: No bootstrap required, opened a new log
I20260925 12:25:41.914577 22057 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260925 12:25:41.914742 22057 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:41.914798 22057 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef2ebcb273c4ace93f47c79902f1331, State: Initialized, Role: FOLLOWER
I20260925 12:25:41.914999 22057 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260925 12:25:41.915102 22057 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:41.915143 22057 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:41.915194 22057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:41.915709 22057 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260925 12:25:41.915831 22057 leader_election.cc:302] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef2ebcb273c4ace93f47c79902f1331; no voters:
I20260925 12:25:41.916021 22057 leader_election.cc:288] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:41.916083 22060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:41.916240 22060 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [term 1 LEADER]: Becoming Leader. State: Replica: eef2ebcb273c4ace93f47c79902f1331, State: Running, Role: LEADER
I20260925 12:25:41.916272 22057 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:41.916347 22060 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260925 12:25:41.916707 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef2ebcb273c4ace93f47c79902f1331. Latest consensus state: current_term: 1 leader_uuid: "eef2ebcb273c4ace93f47c79902f1331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260925 12:25:41.916802 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:41.916703 22061 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef2ebcb273c4ace93f47c79902f1331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2ebcb273c4ace93f47c79902f1331" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260925 12:25:41.916944 22061 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:41.916996 22064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:41.917511 22064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:41.917837 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 21989
I20260925 12:25:41.917945 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:41.918121 1214 external_mini_cluster.cc:1428] Setting key f962431abde266a7f070b1ff7edfa91e
I20260925 12:25:41.919656 22064 catalog_manager.cc:1347] Generated new cluster ID: bbc684a0fc6e4579aa25f134e792a5a5
I20260925 12:25:41.919709 22064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:41.924451 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:41.918558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48797 (local address 127.0.0.1:43935)
0925 12:25:41.919008 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:41.919017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:41.919048 (+ 31us) server_negotiation.cc:407] Connection header received
0925 12:25:41.919161 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:41.919168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:41.919198 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:41.919338 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:41.923987 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:41.923990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:41.923990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:41.923994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:41.924033 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:41.924045 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:41.924120 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:41.924209 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 12:25:41.924254 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":63,"threads_started":1}
I20260925 12:25:41.925558 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:41.929306 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:41.926955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35651 (local address 127.0.0.1:43935)
0925 12:25:41.926991 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:41.926993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:41.928154 (+ 1161us) server_negotiation.cc:407] Connection header received
0925 12:25:41.928212 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:41.928213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:41.928221 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:41.928286 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:41.929017 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:41.929019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:41.929019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:41.929024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:41.929041 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:41.929046 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:41.929119 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:41.929189 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 12:25:41.929208 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:25:41.939769 22064 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:41.939986 22064 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:41.944362 22064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef2ebcb273c4ace93f47c79902f1331: Generated new TSK 0
I20260925 12:25:41.944564 22064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:41.950349 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 21989
2026-09-25T12:25:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:42 dist-test-slave-l9ww krb5kdc[22088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:42 dist-test-slave-l9ww krb5kdc[22088](info): set up 2 sockets
Sep 25 12:25:42 dist-test-slave-l9ww krb5kdc[22088](info): commencing operation
krb5kdc: starting...
W20260925 12:25:42.976152 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:43.096570 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:43.179311 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:43.179529 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:43.179551 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:43.180863 22101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:43.180913 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:43.180927 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:43.180940 22101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:43.180987 22101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:43.182518 22101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:43.182791 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:43.183027 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:43.187901 22106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:43.187906 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:43.187947 22107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:43.188160 22101 server_base.cc:1034] running on GCE node
I20260925 12:25:43.188410 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:43.188724 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:43.189882 22101 hybrid_clock.cc:648] HybridClock initialized: now 1790339143189874 us; error 28 us; skew 500 ppm
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339143, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:43.192382 22101 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:43.192747 22101 webserver.cc:460] Webserver started at http://127.0.0.1:37517/ using document root <none> and password file <none>
I20260925 12:25:43.192994 22101 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:43.193049 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:43.193225 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:43.194286 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "1b87762a9d30452791cac41970e3e9c8"
format_stamp: "Formatted at 2026-09-25 12:25:43 on dist-test-slave-l9ww"
server_key: "1f32ef1f7b2daf85346aef916f22b563"
server_key_iv: "dc43334f24bc98839c3b05217d714643"
server_key_version: "encryptionkey@0"
I20260925 12:25:43.194648 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "1b87762a9d30452791cac41970e3e9c8"
format_stamp: "Formatted at 2026-09-25 12:25:43 on dist-test-slave-l9ww"
server_key: "1f32ef1f7b2daf85346aef916f22b563"
server_key_iv: "dc43334f24bc98839c3b05217d714643"
server_key_version: "encryptionkey@0"
I20260925 12:25:43.196098 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 12:25:43.196827 22116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:43.197005 22101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:43.197077 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "1b87762a9d30452791cac41970e3e9c8"
format_stamp: "Formatted at 2026-09-25 12:25:43 on dist-test-slave-l9ww"
server_key: "1f32ef1f7b2daf85346aef916f22b563"
server_key_iv: "dc43334f24bc98839c3b05217d714643"
server_key_version: "encryptionkey@0"
I20260925 12:25:43.197177 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:43.214008 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:43.216048 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:43.216212 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:43.220526 22101 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33955
I20260925 12:25:43.220593 22168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33955 every 8 connection(s)
I20260925 12:25:43.220937 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:43.221451 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22101
I20260925 12:25:43.221581 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:43.221755 1214 external_mini_cluster.cc:1428] Setting key 3518c535510785af1e40c5bb45089f49
I20260925 12:25:43.222133 22169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:43.225113 22169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8: Bootstrap starting.
I20260925 12:25:43.225752 22169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:43.226078 22169 log.cc:824] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:43.226977 22169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8: No bootstrap required, opened a new log
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 12:25:43.229362 22169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } }
I20260925 12:25:43.229549 22169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:43.229590 22169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b87762a9d30452791cac41970e3e9c8, State: Initialized, Role: FOLLOWER
I20260925 12:25:43.229737 22169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } }
I20260925 12:25:43.229827 22169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:43.229861 22169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:43.229911 22169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:43.230696 22169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } }
I20260925 12:25:43.230811 22169 leader_election.cc:302] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b87762a9d30452791cac41970e3e9c8; no voters:
I20260925 12:25:43.230980 22169 leader_election.cc:288] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:43.231091 22174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:43.231264 22174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [term 1 LEADER]: Becoming Leader. State: Replica: 1b87762a9d30452791cac41970e3e9c8, State: Running, Role: LEADER
I20260925 12:25:43.231294 22169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:43.231364 22174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } }
I20260925 12:25:43.231716 22174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b87762a9d30452791cac41970e3e9c8. Latest consensus state: current_term: 1 leader_uuid: "1b87762a9d30452791cac41970e3e9c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } } }
I20260925 12:25:43.231819 22174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:43.231731 22175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b87762a9d30452791cac41970e3e9c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b87762a9d30452791cac41970e3e9c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33955 } } }
I20260925 12:25:43.232012 22175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:43.232148 22182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:43.232714 22182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:43.233330 22172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:43.222299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:40713 (local address 127.0.0.1:33955)
0925 12:25:43.222496 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:43.222504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:43.222529 (+ 25us) server_negotiation.cc:407] Connection header received
0925 12:25:43.222954 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:43.222961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:43.223009 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:43.223160 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:43.223762 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:43.224725 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:43.225576 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:43.226038 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:43.226163 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:43.228931 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:43.228937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:43.228937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:43.228953 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:43.231013 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:43.231525 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:43.231528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:43.231529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:43.231630 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:43.231933 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:43.231934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:43.231934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:43.232303 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:43.232474 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:43.233046 (+ 572us) server_negotiation.cc:299] Negotiation successful
0925 12:25:43.233157 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260925 12:25:43.234431 22182 catalog_manager.cc:1347] Generated new cluster ID: 46c1264033744d59b3ab761055110604
I20260925 12:25:43.234488 22182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:43.234467 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:43.242461 22172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:43.235823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:35457 (local address 127.0.0.1:33955)
0925 12:25:43.235877 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:43.235879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:43.236002 (+ 123us) server_negotiation.cc:407] Connection header received
0925 12:25:43.236070 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:43.236071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:43.236080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:43.236132 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:43.236639 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:43.237132 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:43.237858 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:43.238082 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:43.238168 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:43.239500 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:43.239502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:43.239503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:43.239510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:43.241090 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:43.241566 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:43.241567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:43.241568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:43.241634 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:43.241887 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:43.241888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:43.241888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:43.242036 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:43.242145 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:43.242292 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 12:25:43.242364 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:25:43.247936 22182 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:43.248675 22182 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:43.253835 22182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b87762a9d30452791cac41970e3e9c8: Generated new TSK 0
I20260925 12:25:43.254045 22182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:43.258885 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22101
W20260925 12:25:43.262993 22086 connection.cc:535] client connection to 127.0.0.1:33955 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33955): Connection reset by peer (error 104)
2026-09-25T12:25:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22200](info): set up 2 sockets
Sep 25 12:25:43 dist-test-slave-l9ww krb5kdc[22200](info): commencing operation
krb5kdc: starting...
W20260925 12:25:44.296190 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:44 dist-test-slave-l9ww krb5kdc[22200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:44.422817 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:44.506846 22213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:44.507078 22213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:44.507102 22213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:44.508342 22213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:44.508387 22213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:44.508401 22213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:44.508415 22213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:44.508427 22213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:44.509913 22213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:44.510129 22213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:44.510304 22213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:44.515470 22218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:44.515484 22219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:44.515484 22221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:44.515558 22213 server_base.cc:1034] running on GCE node
I20260925 12:25:44.516019 22213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:44.516356 22213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:44.517532 22213 hybrid_clock.cc:648] HybridClock initialized: now 1790339144517517 us; error 47 us; skew 500 ppm
Sep 25 12:25:44 dist-test-slave-l9ww krb5kdc[22200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:44.520191 22213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:44.520735 22213 webserver.cc:460] Webserver started at http://127.0.0.1:44049/ using document root <none> and password file <none>
I20260925 12:25:44.521051 22213 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:44.521127 22213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:44.521282 22213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:44.522392 22213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "d73f18f56577479692e1070efa9dcc5b"
format_stamp: "Formatted at 2026-09-25 12:25:44 on dist-test-slave-l9ww"
server_key: "4303daeed502feb8873e117843994566"
server_key_iv: "3c994dc7c79badec85191768b88ecdec"
server_key_version: "encryptionkey@0"
I20260925 12:25:44.522784 22213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "d73f18f56577479692e1070efa9dcc5b"
format_stamp: "Formatted at 2026-09-25 12:25:44 on dist-test-slave-l9ww"
server_key: "4303daeed502feb8873e117843994566"
server_key_iv: "3c994dc7c79badec85191768b88ecdec"
server_key_version: "encryptionkey@0"
I20260925 12:25:44.524286 22213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:44.525228 22228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:44.525430 22213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:44.525559 22213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "d73f18f56577479692e1070efa9dcc5b"
format_stamp: "Formatted at 2026-09-25 12:25:44 on dist-test-slave-l9ww"
server_key: "4303daeed502feb8873e117843994566"
server_key_iv: "3c994dc7c79badec85191768b88ecdec"
server_key_version: "encryptionkey@0"
I20260925 12:25:44.525718 22213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:44.563086 22213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:44.566116 22213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:44.566332 22213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:44.571403 22213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37867
I20260925 12:25:44.571439 22280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37867 every 8 connection(s)
I20260925 12:25:44.571916 22213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:44.572986 22281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:44.575289 22281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b: Bootstrap starting.
I20260925 12:25:44.575837 22281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:44.576175 22281 log.cc:824] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:44.576918 22281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b: No bootstrap required, opened a new log
I20260925 12:25:44.578653 22281 raft_consensus.cc:348] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } }
I20260925 12:25:44.578830 22281 raft_consensus.cc:374] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:44.578878 22281 raft_consensus.cc:729] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73f18f56577479692e1070efa9dcc5b, State: Initialized, Role: FOLLOWER
I20260925 12:25:44.579078 22281 consensus_queue.cc:260] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } }
I20260925 12:25:44.579185 22281 raft_consensus.cc:388] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:44.579231 22281 raft_consensus.cc:482] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:44.579285 22281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:44.579540 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22213
I20260925 12:25:44.579628 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:44.579830 1214 external_mini_cluster.cc:1428] Setting key 6929f0c4ff28d492ad143b5269b36f4c
I20260925 12:25:44.580049 22281 raft_consensus.cc:504] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } }
I20260925 12:25:44.580173 22281 leader_election.cc:302] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d73f18f56577479692e1070efa9dcc5b; no voters:
I20260925 12:25:44.580389 22281 leader_election.cc:288] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:44.580416 22284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:44.580547 22284 raft_consensus.cc:686] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [term 1 LEADER]: Becoming Leader. State: Replica: d73f18f56577479692e1070efa9dcc5b, State: Running, Role: LEADER
I20260925 12:25:44.580639 22284 consensus_queue.cc:237] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } }
I20260925 12:25:44.580682 22281 sys_catalog.cc:564] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:44.581015 22287 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d73f18f56577479692e1070efa9dcc5b. Latest consensus state: current_term: 1 leader_uuid: "d73f18f56577479692e1070efa9dcc5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } } }
I20260925 12:25:44.581251 22287 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:44.581190 22288 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d73f18f56577479692e1070efa9dcc5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73f18f56577479692e1070efa9dcc5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37867 } } }
I20260925 12:25:44.581485 22288 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:44.581768 22291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:44.582265 22291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:44.586359 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:44.580401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:41843 (local address 127.0.0.1:37867)
0925 12:25:44.580603 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:44.580613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:44.580814 (+ 201us) server_negotiation.cc:407] Connection header received
0925 12:25:44.581161 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:44.581169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:44.581234 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:44.581357 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:44.581739 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:44.583207 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:44.584190 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:44.584787 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:44.584900 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:44.585602 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:44.585606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:44.585607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:44.585846 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:44.585891 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:44.585906 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:44.586028 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:44.586111 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 12:25:44.586139 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260925 12:25:44.587393 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:44.587934 22291 catalog_manager.cc:1347] Generated new cluster ID: c74d93b7184d47309e257582a023d21f
I20260925 12:25:44.587996 22291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:44.591876 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:44.588898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:47853 (local address 127.0.0.1:37867)
0925 12:25:44.588935 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:44.588936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:44.589058 (+ 122us) server_negotiation.cc:407] Connection header received
0925 12:25:44.589084 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:44.589084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:44.589094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:44.589158 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:44.589270 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:44.589969 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:44.590637 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:44.590939 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:44.590995 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:44.591646 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:44.591648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:44.591648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:44.591651 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:44.591663 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:44.591667 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:44.591723 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:44.591773 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 12:25:44.591783 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:44.595396 22291 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:44.595981 22291 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:44.602423 22291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d73f18f56577479692e1070efa9dcc5b: Generated new TSK 0
I20260925 12:25:44.602634 22291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:44.609972 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22213
W20260925 12:25:44.614326 22198 connection.cc:535] client connection to 127.0.0.1:37867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37867): Connection reset by peer (error 104)
2026-09-25T12:25:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:44 dist-test-slave-l9ww krb5kdc[22312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:44 dist-test-slave-l9ww krb5kdc[22312](info): set up 2 sockets
Sep 25 12:25:44 dist-test-slave-l9ww krb5kdc[22312](info): commencing operation
krb5kdc: starting...
W20260925 12:25:45.644511 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:45 dist-test-slave-l9ww krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:45.771085 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:45.854329 22325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:45.854542 22325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:45.854565 22325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:45.855674 22325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:45.855715 22325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:45.855729 22325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:45.855742 22325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:45.855754 22325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:45.857250 22325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:45.857448 22325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:45.857635 22325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:45.862778 22333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:45.862923 22325 server_base.cc:1034] running on GCE node
W20260925 12:25:45.862789 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:45.862787 22330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:45.863229 22325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:45.863549 22325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:45.864689 22325 hybrid_clock.cc:648] HybridClock initialized: now 1790339145864680 us; error 31 us; skew 500 ppm
Sep 25 12:25:45 dist-test-slave-l9ww krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:45.867290 22325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:45.867787 22325 webserver.cc:460] Webserver started at http://127.0.0.1:33939/ using document root <none> and password file <none>
I20260925 12:25:45.868077 22325 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:45.868149 22325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:45.868319 22325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:45.869423 22325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "965a5f05ca3543d599890cbd758d77e3"
format_stamp: "Formatted at 2026-09-25 12:25:45 on dist-test-slave-l9ww"
server_key: "07ade9d4891c99f6c4df5c96559621dd"
server_key_iv: "e4cbda6daf83c9425e73f38f49c16c8c"
server_key_version: "encryptionkey@0"
I20260925 12:25:45.869817 22325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "965a5f05ca3543d599890cbd758d77e3"
format_stamp: "Formatted at 2026-09-25 12:25:45 on dist-test-slave-l9ww"
server_key: "07ade9d4891c99f6c4df5c96559621dd"
server_key_iv: "e4cbda6daf83c9425e73f38f49c16c8c"
server_key_version: "encryptionkey@0"
I20260925 12:25:45.871220 22325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:45.872066 22340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:45.872262 22325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:45.872373 22325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "965a5f05ca3543d599890cbd758d77e3"
format_stamp: "Formatted at 2026-09-25 12:25:45 on dist-test-slave-l9ww"
server_key: "07ade9d4891c99f6c4df5c96559621dd"
server_key_iv: "e4cbda6daf83c9425e73f38f49c16c8c"
server_key_version: "encryptionkey@0"
I20260925 12:25:45.872452 22325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:45.875826 22325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:45.878255 22325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:45.878365 22325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:45.882596 22325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43741
I20260925 12:25:45.882632 22392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43741 every 8 connection(s)
I20260925 12:25:45.882973 22325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:45.884013 22393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:45.886364 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3: Bootstrap starting.
I20260925 12:25:45.886482 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22325
I20260925 12:25:45.886582 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:45.886790 1214 external_mini_cluster.cc:1428] Setting key 2d87c3fea336b3dceef576bc7fbc0bf7
I20260925 12:25:45.887127 22393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:45.887454 22393 log.cc:824] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:45.888339 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3: No bootstrap required, opened a new log
I20260925 12:25:45.889621 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:45.887282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:56899 (local address 127.0.0.1:43741)
0925 12:25:45.887512 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:45.887523 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:45.887562 (+ 39us) server_negotiation.cc:407] Connection header received
0925 12:25:45.888219 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:45.888227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:45.888260 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:45.888421 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:45.889126 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:45.889130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:45.889130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:45.889134 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:45.889187 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:45.889198 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:45.889290 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:45.889360 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 12:25:45.889460 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260925 12:25:45.890671 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:45.890909 22393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } }
I20260925 12:25:45.891072 22393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:45.891124 22393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965a5f05ca3543d599890cbd758d77e3, State: Initialized, Role: FOLLOWER
I20260925 12:25:45.891251 22393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } }
I20260925 12:25:45.891343 22393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:45.891387 22393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:45.891438 22393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:45.892303 22393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } }
I20260925 12:25:45.892427 22393 leader_election.cc:302] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965a5f05ca3543d599890cbd758d77e3; no voters:
I20260925 12:25:45.892618 22393 leader_election.cc:288] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:45.892663 22403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:45.892810 22403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [term 1 LEADER]: Becoming Leader. State: Replica: 965a5f05ca3543d599890cbd758d77e3, State: Running, Role: LEADER
I20260925 12:25:45.892915 22393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:45.892942 22403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } }
I20260925 12:25:45.893303 22405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "965a5f05ca3543d599890cbd758d77e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } } }
I20260925 12:25:45.893395 22405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:45.893404 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:45.892200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:57007 (local address 127.0.0.1:43741)
0925 12:25:45.892270 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:45.892272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:45.892339 (+ 67us) server_negotiation.cc:407] Connection header received
0925 12:25:45.892404 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:45.892405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:45.892410 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:45.892471 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:45.893084 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:45.893085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:45.893086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:45.893090 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:45.893103 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:45.893106 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:45.893154 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:45.893331 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 12:25:45.893346 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 12:25:45.893471 22404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 965a5f05ca3543d599890cbd758d77e3. Latest consensus state: current_term: 1 leader_uuid: "965a5f05ca3543d599890cbd758d77e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965a5f05ca3543d599890cbd758d77e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43741 } } }
I20260925 12:25:45.893867 22410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:45.893980 22404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:45.894510 22410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:45.896230 22410 catalog_manager.cc:1347] Generated new cluster ID: 84ec3cee871e4738aced608158519158
I20260925 12:25:45.896282 22410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:45.911206 22410 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:45.911403 22410 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:45.916587 22410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 965a5f05ca3543d599890cbd758d77e3: Generated new TSK 0
I20260925 12:25:45.916800 22410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:45.923127 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22325
2026-09-25T12:25:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:45 dist-test-slave-l9ww krb5kdc[22424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:45 dist-test-slave-l9ww krb5kdc[22424](info): set up 2 sockets
Sep 25 12:25:45 dist-test-slave-l9ww krb5kdc[22424](info): commencing operation
krb5kdc: starting...
W20260925 12:25:46.956169 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:47.083720 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37587
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:47.170797 22437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:47.171056 22437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:47.171098 22437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:47.172312 22437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:47.172384 22437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:47.172408 22437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:47.172431 22437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:47.172461 22437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:47.173974 22437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:47.174269 22437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:47.174508 22437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:47.179409 22445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:47.179409 22443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:47.179418 22442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:47.179643 22437 server_base.cc:1034] running on GCE node
I20260925 12:25:47.180018 22437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:47.180392 22437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:47.181584 22437 hybrid_clock.cc:648] HybridClock initialized: now 1790339147181571 us; error 32 us; skew 500 ppm
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:47.190829 22437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:47.191303 22437 webserver.cc:460] Webserver started at http://127.0.0.1:42765/ using document root <none> and password file <none>
I20260925 12:25:47.191578 22437 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:47.191628 22437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:47.191797 22437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:47.193030 22437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "ef1054843770462bb223dbf184eb2724"
format_stamp: "Formatted at 2026-09-25 12:25:47 on dist-test-slave-l9ww"
server_key: "9482fdf314cf9d61206095ea0e7f4e56"
server_key_iv: "0ce78e74c788b9b6f863a1dc4b556fe0"
server_key_version: "encryptionkey@0"
I20260925 12:25:47.193413 22437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "ef1054843770462bb223dbf184eb2724"
format_stamp: "Formatted at 2026-09-25 12:25:47 on dist-test-slave-l9ww"
server_key: "9482fdf314cf9d61206095ea0e7f4e56"
server_key_iv: "0ce78e74c788b9b6f863a1dc4b556fe0"
server_key_version: "encryptionkey@0"
I20260925 12:25:47.194898 22437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:47.195852 22452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:47.196159 22437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:47.196313 22437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "ef1054843770462bb223dbf184eb2724"
format_stamp: "Formatted at 2026-09-25 12:25:47 on dist-test-slave-l9ww"
server_key: "9482fdf314cf9d61206095ea0e7f4e56"
server_key_iv: "0ce78e74c788b9b6f863a1dc4b556fe0"
server_key_version: "encryptionkey@0"
I20260925 12:25:47.196408 22437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:47.228230 22437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:47.231027 22437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:47.231237 22437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:47.235844 22437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37587
I20260925 12:25:47.235962 22504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37587 every 8 connection(s)
I20260925 12:25:47.236344 22437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:47.237407 22505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:47.239675 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22437
I20260925 12:25:47.239733 22505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724: Bootstrap starting.
I20260925 12:25:47.239799 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:47.239985 1214 external_mini_cluster.cc:1428] Setting key bea8d7d93ee5b74b0a4abfc02455647c
I20260925 12:25:47.240482 22505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:47.240835 22505 log.cc:824] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:47.241592 22505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724: No bootstrap required, opened a new log
I20260925 12:25:47.243932 22505 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } }
I20260925 12:25:47.244117 22505 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:47.244150 22505 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1054843770462bb223dbf184eb2724, State: Initialized, Role: FOLLOWER
I20260925 12:25:47.244273 22505 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } }
I20260925 12:25:47.244340 22505 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:47.244366 22505 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:47.244410 22505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:47.244994 22505 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } }
I20260925 12:25:47.245126 22505 leader_election.cc:302] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef1054843770462bb223dbf184eb2724; no voters:
I20260925 12:25:47.245303 22505 leader_election.cc:288] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:47.245323 22510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:47.245487 22510 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [term 1 LEADER]: Becoming Leader. State: Replica: ef1054843770462bb223dbf184eb2724, State: Running, Role: LEADER
I20260925 12:25:47.245570 22510 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } }
I20260925 12:25:47.245600 22505 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:47.246039 22512 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef1054843770462bb223dbf184eb2724. Latest consensus state: current_term: 1 leader_uuid: "ef1054843770462bb223dbf184eb2724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } } }
I20260925 12:25:47.246068 22511 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef1054843770462bb223dbf184eb2724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1054843770462bb223dbf184eb2724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37587 } } }
I20260925 12:25:47.246135 22512 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:47.246142 22511 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:47.246395 22518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:47.246863 22518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:47.248425 22518 catalog_manager.cc:1347] Generated new cluster ID: 8324bc8bd117497eaf9fd4ff34d6a1f2
I20260925 12:25:47.248492 22518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 12:25:47.256348 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:47.240541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:40277 (local address 127.0.0.1:37587)
0925 12:25:47.240738 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:47.240747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:47.240774 (+ 27us) server_negotiation.cc:407] Connection header received
0925 12:25:47.241494 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:47.241504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:47.241564 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:47.241783 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:47.242547 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:47.243463 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:47.244585 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:47.245005 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:47.245124 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:47.251756 (+ 6632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:47.251761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:47.251761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:47.251777 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:47.254445 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:47.254946 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:47.254950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:47.254951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:47.255073 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:47.255323 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:47.255324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:47.255325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:47.255661 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:47.255832 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:47.255997 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 12:25:47.256112 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260925 12:25:47.257728 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:47.267036 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:47.259440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:35701 (local address 127.0.0.1:37587)
0925 12:25:47.259480 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:47.259481 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:47.259625 (+ 144us) server_negotiation.cc:407] Connection header received
0925 12:25:47.259684 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:47.259685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:47.259694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:47.259741 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:47.260438 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:47.261183 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:47.262224 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:47.262548 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:47.262609 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:47.264046 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:47.264047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:47.264048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:47.264055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:47.265664 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:47.266142 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:47.266143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:47.266143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:47.266211 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:47.266482 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:47.266482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:47.266482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:47.266617 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:47.266713 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:47.266872 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 12:25:47.266938 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:25:47.268802 22518 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:47.269594 22518 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:47.277985 22518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef1054843770462bb223dbf184eb2724: Generated new TSK 0
I20260925 12:25:47.278237 22518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:47.281983 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22437
W20260925 12:25:47.285817 22422 connection.cc:535] client connection to 127.0.0.1:37587 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37587): Connection reset by peer (error 104)
2026-09-25T12:25:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22536](info): set up 2 sockets
Sep 25 12:25:47 dist-test-slave-l9ww krb5kdc[22536](info): commencing operation
krb5kdc: starting...
W20260925 12:25:48.328070 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:48 dist-test-slave-l9ww krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:48.459739 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:48.544831 22549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:48.545049 22549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:48.545081 22549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:48.546222 22549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:48.546260 22549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:48.546274 22549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:48.546288 22549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:48.546299 22549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:48.547755 22549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:48.547991 22549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:48.548175 22549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:48.552969 22554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:48.552969 22557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:48.553009 22555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:48.553368 22549 server_base.cc:1034] running on GCE node
I20260925 12:25:48.553853 22549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:48.554160 22549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:48.555334 22549 hybrid_clock.cc:648] HybridClock initialized: now 1790339148555316 us; error 29 us; skew 500 ppm
Sep 25 12:25:48 dist-test-slave-l9ww krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:48.558301 22549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:48.558836 22549 webserver.cc:460] Webserver started at http://127.0.0.1:39873/ using document root <none> and password file <none>
I20260925 12:25:48.559099 22549 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:48.559147 22549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:48.559317 22549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:48.560478 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "caaaef00204e497a8bd775c7e58ab064"
format_stamp: "Formatted at 2026-09-25 12:25:48 on dist-test-slave-l9ww"
server_key: "196fe631cdb76a160a3014fad6baddf4"
server_key_iv: "76889e61dc65aaa67dc2f7b74e2e29d8"
server_key_version: "encryptionkey@0"
I20260925 12:25:48.560858 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "caaaef00204e497a8bd775c7e58ab064"
format_stamp: "Formatted at 2026-09-25 12:25:48 on dist-test-slave-l9ww"
server_key: "196fe631cdb76a160a3014fad6baddf4"
server_key_iv: "76889e61dc65aaa67dc2f7b74e2e29d8"
server_key_version: "encryptionkey@0"
I20260925 12:25:48.562253 22549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:48.563060 22564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:48.563318 22549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:48.563424 22549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "caaaef00204e497a8bd775c7e58ab064"
format_stamp: "Formatted at 2026-09-25 12:25:48 on dist-test-slave-l9ww"
server_key: "196fe631cdb76a160a3014fad6baddf4"
server_key_iv: "76889e61dc65aaa67dc2f7b74e2e29d8"
server_key_version: "encryptionkey@0"
I20260925 12:25:48.563488 22549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:48.574717 22549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:48.577200 22549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:48.577338 22549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:48.581696 22549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36573
I20260925 12:25:48.581730 22616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36573 every 8 connection(s)
I20260925 12:25:48.582124 22549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:48.583093 22617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:48.585335 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22549
I20260925 12:25:48.585456 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:48.585628 1214 external_mini_cluster.cc:1428] Setting key 3345cc1be79d403c201a3ed0fc90f7de
I20260925 12:25:48.586273 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064: Bootstrap starting.
I20260925 12:25:48.586959 22617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:48.587293 22617 log.cc:824] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:48.588155 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064: No bootstrap required, opened a new log
I20260925 12:25:48.590673 22617 raft_consensus.cc:348] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } }
I20260925 12:25:48.590873 22617 raft_consensus.cc:374] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:48.590914 22617 raft_consensus.cc:729] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caaaef00204e497a8bd775c7e58ab064, State: Initialized, Role: FOLLOWER
I20260925 12:25:48.591094 22617 consensus_queue.cc:260] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } }
I20260925 12:25:48.591173 22617 raft_consensus.cc:388] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:48.591198 22617 raft_consensus.cc:482] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:48.591262 22617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:48.592209 22617 raft_consensus.cc:504] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } }
I20260925 12:25:48.592334 22617 leader_election.cc:302] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caaaef00204e497a8bd775c7e58ab064; no voters:
I20260925 12:25:48.592573 22617 leader_election.cc:288] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:48.592608 22622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:48.592792 22622 raft_consensus.cc:686] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [term 1 LEADER]: Becoming Leader. State: Replica: caaaef00204e497a8bd775c7e58ab064, State: Running, Role: LEADER
I20260925 12:25:48.592887 22617 sys_catalog.cc:564] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:48.592880 22622 consensus_queue.cc:237] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } }
I20260925 12:25:48.593256 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:48.586325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:34985 (local address 127.0.0.1:36573)
0925 12:25:48.586555 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:48.586570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:48.586599 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:25:48.587136 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:48.587144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:48.587201 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:48.587365 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:48.587612 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:48.589171 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:48.590009 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:48.590779 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:48.590911 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:48.591791 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:48.591795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:48.591796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:48.592063 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:48.592109 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:48.592137 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:48.592241 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:48.592535 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 12:25:48.592580 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
I20260925 12:25:48.593438 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caaaef00204e497a8bd775c7e58ab064. Latest consensus state: current_term: 1 leader_uuid: "caaaef00204e497a8bd775c7e58ab064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } } }
I20260925 12:25:48.593521 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:48.593600 22624 sys_catalog.cc:455] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caaaef00204e497a8bd775c7e58ab064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caaaef00204e497a8bd775c7e58ab064" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36573 } } }
I20260925 12:25:48.593727 22627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:48.593751 22624 sys_catalog.cc:458] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:48.594223 22627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:48.595521 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:48.596624 22627 catalog_manager.cc:1347] Generated new cluster ID: 79ba11f29e9a4e2aa89f3f848bcdedc4
I20260925 12:25:48.596693 22627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:48.601428 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:48.597219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:37801 (local address 127.0.0.1:36573)
0925 12:25:48.597261 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:48.597263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:48.597396 (+ 133us) server_negotiation.cc:407] Connection header received
0925 12:25:48.597464 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:48.597465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:48.597476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:48.597539 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:48.597707 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:48.598530 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:48.599756 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:48.600204 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:48.600273 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:48.601130 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:48.601131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:48.601131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:48.601137 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:48.601153 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:48.601157 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:48.601218 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:48.601326 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 12:25:48.601338 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:48.626744 22627 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:48.627563 22627 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:48.640491 22627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caaaef00204e497a8bd775c7e58ab064: Generated new TSK 0
I20260925 12:25:48.640781 22627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:48.652453 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22549
W20260925 12:25:48.656939 22534 connection.cc:535] client connection to 127.0.0.1:36573 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36573): Connection reset by peer (error 104)
2026-09-25T12:25:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:48 dist-test-slave-l9ww krb5kdc[22648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:48 dist-test-slave-l9ww krb5kdc[22648](info): set up 2 sockets
Sep 25 12:25:48 dist-test-slave-l9ww krb5kdc[22648](info): commencing operation
krb5kdc: starting...
W20260925 12:25:49.704217 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:49 dist-test-slave-l9ww krb5kdc[22648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 12:25:49.831008 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:49.916689 22661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:49.916896 22661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:49.916918 22661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:49.918082 22661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:49.918125 22661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:49.918140 22661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:49.918154 22661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:49.918164 22661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:49.919659 22661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:49.919948 22661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:49.920198 22661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:49.925120 22666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:49.925246 22661 server_base.cc:1034] running on GCE node
W20260925 12:25:49.925069 22669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:49.925069 22667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:49.925580 22661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:49.925935 22661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:49.927100 22661 hybrid_clock.cc:648] HybridClock initialized: now 1790339149927075 us; error 51 us; skew 500 ppm
Sep 25 12:25:49 dist-test-slave-l9ww krb5kdc[22648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:49.929873 22661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 12:25:49.930424 22661 webserver.cc:460] Webserver started at http://127.0.0.1:34785/ using document root <none> and password file <none>
I20260925 12:25:49.930680 22661 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:49.930751 22661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:49.930900 22661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:49.932101 22661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "3fd4d017c2214fcc896b126b39cd4f43"
format_stamp: "Formatted at 2026-09-25 12:25:49 on dist-test-slave-l9ww"
server_key: "6ff3c42bf45a13e12acd0305135359bf"
server_key_iv: "015f7e12a4fe8df63fd05e184bcab325"
server_key_version: "encryptionkey@0"
I20260925 12:25:49.932462 22661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "3fd4d017c2214fcc896b126b39cd4f43"
format_stamp: "Formatted at 2026-09-25 12:25:49 on dist-test-slave-l9ww"
server_key: "6ff3c42bf45a13e12acd0305135359bf"
server_key_iv: "015f7e12a4fe8df63fd05e184bcab325"
server_key_version: "encryptionkey@0"
I20260925 12:25:49.933854 22661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:49.934749 22676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:49.935010 22661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:49.935153 22661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "3fd4d017c2214fcc896b126b39cd4f43"
format_stamp: "Formatted at 2026-09-25 12:25:49 on dist-test-slave-l9ww"
server_key: "6ff3c42bf45a13e12acd0305135359bf"
server_key_iv: "015f7e12a4fe8df63fd05e184bcab325"
server_key_version: "encryptionkey@0"
I20260925 12:25:49.935252 22661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:49.938798 22661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:49.941250 22661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:49.941418 22661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:49.945662 22661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41049
I20260925 12:25:49.945693 22728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41049 every 8 connection(s)
I20260925 12:25:49.946090 22661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:49.947147 22729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:49.949503 22729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43: Bootstrap starting.
I20260925 12:25:49.950047 22729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:49.950279 22729 log.cc:824] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:49.950953 22729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43: No bootstrap required, opened a new log
I20260925 12:25:49.952600 22729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } }
I20260925 12:25:49.952755 22729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:49.952778 22729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd4d017c2214fcc896b126b39cd4f43, State: Initialized, Role: FOLLOWER
I20260925 12:25:49.952924 22729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } }
I20260925 12:25:49.953003 22729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:49.953025 22729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:49.953053 22729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:49.953552 22729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } }
I20260925 12:25:49.953645 22729 leader_election.cc:302] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fd4d017c2214fcc896b126b39cd4f43; no voters:
I20260925 12:25:49.953766 22729 leader_election.cc:288] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:49.953859 22732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:49.954049 22732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [term 1 LEADER]: Becoming Leader. State: Replica: 3fd4d017c2214fcc896b126b39cd4f43, State: Running, Role: LEADER
I20260925 12:25:49.954053 22729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:49.954167 22732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } }
I20260925 12:25:49.954610 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fd4d017c2214fcc896b126b39cd4f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } } }
I20260925 12:25:49.954654 22734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fd4d017c2214fcc896b126b39cd4f43. Latest consensus state: current_term: 1 leader_uuid: "3fd4d017c2214fcc896b126b39cd4f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd4d017c2214fcc896b126b39cd4f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41049 } } }
I20260925 12:25:49.954777 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:49.954900 22734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:49.955295 22738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:49.955881 22738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:49.956357 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22661
I20260925 12:25:49.956467 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:49.956669 1214 external_mini_cluster.cc:1428] Setting key 45d9ee01de7039cb00e7292f39797395
I20260925 12:25:49.957867 22738 catalog_manager.cc:1347] Generated new cluster ID: e84662e463c347c6a1ff793208461a66
I20260925 12:25:49.957924 22738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:49.959473 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:49.957387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:32991 (local address 127.0.0.1:41049)
0925 12:25:49.957549 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:49.957558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:49.957592 (+ 34us) server_negotiation.cc:407] Connection header received
0925 12:25:49.957719 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:49.957726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:49.957757 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:49.957900 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:49.958988 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:49.958993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:49.958993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:49.958999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:49.959046 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:49.959063 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:49.959141 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:49.959224 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 12:25:49.959274 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260925 12:25:49.960939 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:49.965111 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:49.962451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:40813 (local address 127.0.0.1:41049)
0925 12:25:49.962487 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:49.962488 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:49.964081 (+ 1593us) server_negotiation.cc:407] Connection header received
0925 12:25:49.964157 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:49.964157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:49.964163 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:49.964222 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:49.964875 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:49.964876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:49.964877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:49.964880 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:49.964895 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:49.964900 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:49.964952 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:49.965028 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 12:25:49.965042 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:25:49.978530 22738 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:49.978782 22738 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:49.983438 22738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fd4d017c2214fcc896b126b39cd4f43: Generated new TSK 0
I20260925 12:25:49.983680 22738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:49.985563 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22661
2026-09-25T12:25:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:50 dist-test-slave-l9ww krb5kdc[22760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:50 dist-test-slave-l9ww krb5kdc[22760](info): set up 2 sockets
Sep 25 12:25:50 dist-test-slave-l9ww krb5kdc[22760](info): commencing operation
krb5kdc: starting...
W20260925 12:25:51.024569 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:51.151556 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36681
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:36681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:51.235440 22773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:51.235668 22773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:51.235692 22773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:51.236943 22773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:51.236990 22773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:51.237004 22773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:51.237017 22773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:51.237030 22773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:51.238448 22773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:36681
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:36681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:51.238668 22773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:51.238837 22773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:51.243747 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:51.243755 22781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:51.243897 22773 server_base.cc:1034] running on GCE node
W20260925 12:25:51.243755 22779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:51.244236 22773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:51.244534 22773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:51.245720 22773 hybrid_clock.cc:648] HybridClock initialized: now 1790339151245701 us; error 41 us; skew 500 ppm
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339151, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:51.248667 22773 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:51.249285 22773 webserver.cc:460] Webserver started at http://127.1.47.190:40331/ using document root <none> and password file <none>
I20260925 12:25:51.249588 22773 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:51.249660 22773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:51.249809 22773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:51.251320 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "c6df841460544952a7dd5ddf827f9e16"
format_stamp: "Formatted at 2026-09-25 12:25:51 on dist-test-slave-l9ww"
server_key: "fa93bb9bb8f453e18432d0f2178f2bd3"
server_key_iv: "a4e5ea206b6d7112c4d7cd91c9306726"
server_key_version: "encryptionkey@0"
I20260925 12:25:51.251804 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "c6df841460544952a7dd5ddf827f9e16"
format_stamp: "Formatted at 2026-09-25 12:25:51 on dist-test-slave-l9ww"
server_key: "fa93bb9bb8f453e18432d0f2178f2bd3"
server_key_iv: "a4e5ea206b6d7112c4d7cd91c9306726"
server_key_version: "encryptionkey@0"
I20260925 12:25:51.253775 22773 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260925 12:25:51.254812 22788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:51.255098 22773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:51.255208 22773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "c6df841460544952a7dd5ddf827f9e16"
format_stamp: "Formatted at 2026-09-25 12:25:51 on dist-test-slave-l9ww"
server_key: "fa93bb9bb8f453e18432d0f2178f2bd3"
server_key_iv: "a4e5ea206b6d7112c4d7cd91c9306726"
server_key_version: "encryptionkey@0"
I20260925 12:25:51.255292 22773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:51.276693 22773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:51.279057 22773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:51.279174 22773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:51.283350 22773 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:36681
I20260925 12:25:51.283383 22840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:36681 every 8 connection(s)
I20260925 12:25:51.283780 22773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:51.284665 22841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:51.286849 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22773
I20260925 12:25:51.286933 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:51.287043 22841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16: Bootstrap starting.
I20260925 12:25:51.287176 1214 external_mini_cluster.cc:1428] Setting key d0b991b192de79cbae18fad83da501f9
I20260925 12:25:51.287825 22841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:51.288234 22841 log.cc:824] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:51.289032 22841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16: No bootstrap required, opened a new log
I20260925 12:25:51.291415 22841 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } }
I20260925 12:25:51.291553 22841 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:51.291576 22841 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6df841460544952a7dd5ddf827f9e16, State: Initialized, Role: FOLLOWER
I20260925 12:25:51.291709 22841 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } }
I20260925 12:25:51.291793 22841 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:51.291846 22841 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:51.291972 22841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:51.292704 22841 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } }
I20260925 12:25:51.292824 22841 leader_election.cc:302] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6df841460544952a7dd5ddf827f9e16; no voters:
I20260925 12:25:51.292985 22841 leader_election.cc:288] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:51.293040 22846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:51.293211 22846 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [term 1 LEADER]: Becoming Leader. State: Replica: c6df841460544952a7dd5ddf827f9e16, State: Running, Role: LEADER
I20260925 12:25:51.293263 22841 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:51.293387 22846 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } }
I20260925 12:25:51.293926 22848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6df841460544952a7dd5ddf827f9e16. Latest consensus state: current_term: 1 leader_uuid: "c6df841460544952a7dd5ddf827f9e16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } } }
I20260925 12:25:51.294023 22848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:51.294337 22847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6df841460544952a7dd5ddf827f9e16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6df841460544952a7dd5ddf827f9e16" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 36681 } } }
I20260925 12:25:51.294500 22847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:51.294693 22857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:51.295190 22857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:51.297039 22857 catalog_manager.cc:1347] Generated new cluster ID: dc1981de288845dfb8c166d7bddc9f46
I20260925 12:25:51.297091 22857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:51.303375 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:51.287828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51705 (local address 127.1.47.190:36681)
0925 12:25:51.288073 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:51.288084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:51.288119 (+ 35us) server_negotiation.cc:407] Connection header received
0925 12:25:51.288705 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:51.288714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:51.288781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:51.288935 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:51.289623 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:51.291086 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:51.292110 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:51.292734 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:51.292853 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:51.295928 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:51.295933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:51.295933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:51.295948 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:51.301485 (+ 5537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:51.302025 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:51.302028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:51.302029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:51.302141 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:51.302411 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:51.302412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:51.302412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:51.302726 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:51.302877 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:51.303043 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 12:25:51.303148 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":117,"threads_started":1}
I20260925 12:25:51.304401 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:51.315927 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:51.308307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55735 (local address 127.1.47.190:36681)
0925 12:25:51.308344 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:51.308346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:51.308463 (+ 117us) server_negotiation.cc:407] Connection header received
0925 12:25:51.308523 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:51.308524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:51.308534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:51.308593 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:51.309250 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:51.310006 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:51.311058 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:51.311355 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:51.311417 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:51.312903 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:51.312904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:51.312905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:51.312913 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:51.314613 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:51.315041 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:51.315042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:51.315043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:51.315104 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:51.315332 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:51.315333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:51.315333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:51.315469 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:51.315576 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:51.315724 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 12:25:51.315792 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:25:51.317214 22857 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:51.317912 22857 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:51.321095 22857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6df841460544952a7dd5ddf827f9e16: Generated new TSK 0
I20260925 12:25:51.321286 22857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:51.323838 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22773
W20260925 12:25:51.328454 22758 connection.cc:535] client connection to 127.1.47.190:36681 recv error: Network error: failed to read from TLS socket (remote: 127.1.47.190:36681): Connection reset by peer (error 104)
2026-09-25T12:25:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22872](info): set up 2 sockets
Sep 25 12:25:51 dist-test-slave-l9ww krb5kdc[22872](info): commencing operation
krb5kdc: starting...
W20260925 12:25:52.352108 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:52 dist-test-slave-l9ww krb5kdc[22872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:52.475396 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:33381
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:52.556715 22885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:52.556985 22885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:52.557026 22885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:52.558230 22885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:52.558315 22885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:52.558339 22885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:52.558362 22885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:52.558389 22885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:52.559948 22885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:33381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:52.560253 22885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:52.560497 22885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:52.565542 22893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:52.565519 22890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:52.565516 22891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:52.565766 22885 server_base.cc:1034] running on GCE node
I20260925 12:25:52.566162 22885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:52.566510 22885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:52.567706 22885 hybrid_clock.cc:648] HybridClock initialized: now 1790339152567683 us; error 35 us; skew 500 ppm
Sep 25 12:25:52 dist-test-slave-l9ww krb5kdc[22872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339152, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:52.573876 22885 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:52.574422 22885 webserver.cc:460] Webserver started at http://127.1.47.190:46409/ using document root <none> and password file <none>
I20260925 12:25:52.574694 22885 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:52.574771 22885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:52.574941 22885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:52.576133 22885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "1c63f32f857a4ab48967d8aabfce404e"
format_stamp: "Formatted at 2026-09-25 12:25:52 on dist-test-slave-l9ww"
server_key: "61ae9e9e7a37a0f5078795dde47071ec"
server_key_iv: "ba835dc34cf952cde6b22f19096b8ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:52.576502 22885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "1c63f32f857a4ab48967d8aabfce404e"
format_stamp: "Formatted at 2026-09-25 12:25:52 on dist-test-slave-l9ww"
server_key: "61ae9e9e7a37a0f5078795dde47071ec"
server_key_iv: "ba835dc34cf952cde6b22f19096b8ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:52.577876 22885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:52.578646 22900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:52.578840 22885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:52.578931 22885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "1c63f32f857a4ab48967d8aabfce404e"
format_stamp: "Formatted at 2026-09-25 12:25:52 on dist-test-slave-l9ww"
server_key: "61ae9e9e7a37a0f5078795dde47071ec"
server_key_iv: "ba835dc34cf952cde6b22f19096b8ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:52.579054 22885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:52.612219 22885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:52.614346 22885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:52.614459 22885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:52.618749 22885 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:33381
I20260925 12:25:52.618788 22952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:33381 every 8 connection(s)
I20260925 12:25:52.619169 22885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:52.620131 22953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:52.620704 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22885
I20260925 12:25:52.620821 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:52.621016 1214 external_mini_cluster.cc:1428] Setting key 4b84b4b4501d8adf2dadbff7ce5a5bc6
I20260925 12:25:52.623172 22953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e: Bootstrap starting.
I20260925 12:25:52.623828 22953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:52.624159 22953 log.cc:824] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:52.624939 22953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e: No bootstrap required, opened a new log
I20260925 12:25:52.626783 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:52.621611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42331 (local address 127.1.47.190:33381)
0925 12:25:52.621797 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:52.621806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:52.621840 (+ 34us) server_negotiation.cc:407] Connection header received
0925 12:25:52.622242 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:52.622248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:52.622295 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:52.622412 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:52.622694 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:52.623718 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:52.624799 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:52.625263 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:52.625344 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:52.626126 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:52.626130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:52.626130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:52.626291 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:52.626321 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:52.626331 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:52.626396 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:52.626514 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 12:25:52.626560 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260925 12:25:52.627395 22953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } }
I20260925 12:25:52.627578 22953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:52.627627 22953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63f32f857a4ab48967d8aabfce404e, State: Initialized, Role: FOLLOWER
I20260925 12:25:52.627739 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:52.627785 22953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } }
I20260925 12:25:52.627892 22953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:52.627930 22953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:52.627975 22953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:52.628765 22953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } }
I20260925 12:25:52.628894 22953 leader_election.cc:302] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c63f32f857a4ab48967d8aabfce404e; no voters:
I20260925 12:25:52.629057 22953 leader_election.cc:288] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:52.629120 22962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:52.629318 22962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [term 1 LEADER]: Becoming Leader. State: Replica: 1c63f32f857a4ab48967d8aabfce404e, State: Running, Role: LEADER
I20260925 12:25:52.629434 22962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } }
I20260925 12:25:52.629319 22953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:52.629838 22964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c63f32f857a4ab48967d8aabfce404e. Latest consensus state: current_term: 1 leader_uuid: "1c63f32f857a4ab48967d8aabfce404e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } } }
I20260925 12:25:52.629932 22964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:52.630157 22963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c63f32f857a4ab48967d8aabfce404e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c63f32f857a4ab48967d8aabfce404e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33381 } } }
I20260925 12:25:52.630259 22963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:52.630400 22967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:52.630877 22967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:52.632733 22967 catalog_manager.cc:1347] Generated new cluster ID: 71917ac01c304b4e9585b593a4049053
I20260925 12:25:52.632838 22967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:52.633126 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:52.629359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37181 (local address 127.1.47.190:33381)
0925 12:25:52.629389 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:52.629391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:52.629567 (+ 176us) server_negotiation.cc:407] Connection header received
0925 12:25:52.629677 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:52.629678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:52.629689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:52.629747 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:52.629908 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:52.630649 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:52.631710 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:52.632023 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:52.632098 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:52.632852 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:52.632854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:52.632854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:52.632859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:52.632875 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:52.632880 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:52.632958 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:52.633043 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 12:25:52.633055 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 12:25:52.664184 22967 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:52.664881 22967 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:52.678748 22967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c63f32f857a4ab48967d8aabfce404e: Generated new TSK 0
I20260925 12:25:52.679014 22967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:52.688659 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22885
W20260925 12:25:52.693262 22870 connection.cc:535] client connection to 127.1.47.190:33381 recv error: Network error: failed to read from TLS socket (remote: 127.1.47.190:33381): Connection reset by peer (error 104)
2026-09-25T12:25:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:52 dist-test-slave-l9ww krb5kdc[22984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:52 dist-test-slave-l9ww krb5kdc[22984](info): set up 2 sockets
Sep 25 12:25:52 dist-test-slave-l9ww krb5kdc[22984](info): commencing operation
krb5kdc: starting...
W20260925 12:25:53.764122 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:53 dist-test-slave-l9ww krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:53.887045 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:33521
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:53.969033 22997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:53.969307 22997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:53.969347 22997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:53.970520 22997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:53.970604 22997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:53.970630 22997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:53.970675 22997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:53.970707 22997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:53.972304 22997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:33521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:53.972592 22997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:53.972826 22997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:53.977631 23005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:53.977631 23002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:53.977872 22997 server_base.cc:1034] running on GCE node
W20260925 12:25:53.977654 23003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:53.978140 22997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:53.978457 22997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:53.979624 22997 hybrid_clock.cc:648] HybridClock initialized: now 1790339153979603 us; error 39 us; skew 500 ppm
Sep 25 12:25:53 dist-test-slave-l9ww krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:53.982227 22997 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:53.982717 22997 webserver.cc:460] Webserver started at http://127.1.47.190:44413/ using document root <none> and password file <none>
I20260925 12:25:53.982956 22997 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:53.983024 22997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:53.983162 22997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:53.984328 22997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "d659d9393e2c4a108e9449a9ac3ec61d"
format_stamp: "Formatted at 2026-09-25 12:25:53 on dist-test-slave-l9ww"
server_key: "9b3fcd5ff17c42ab9c13c48e9461a289"
server_key_iv: "980a9137c2ae19dc41ef1474689783a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:53.984706 22997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "d659d9393e2c4a108e9449a9ac3ec61d"
format_stamp: "Formatted at 2026-09-25 12:25:53 on dist-test-slave-l9ww"
server_key: "9b3fcd5ff17c42ab9c13c48e9461a289"
server_key_iv: "980a9137c2ae19dc41ef1474689783a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:53.986038 22997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:53.986761 23012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:53.986934 22997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:53.987037 22997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "d659d9393e2c4a108e9449a9ac3ec61d"
format_stamp: "Formatted at 2026-09-25 12:25:53 on dist-test-slave-l9ww"
server_key: "9b3fcd5ff17c42ab9c13c48e9461a289"
server_key_iv: "980a9137c2ae19dc41ef1474689783a1"
server_key_version: "encryptionkey@0"
I20260925 12:25:53.987119 22997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:53.990491 22997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:53.992389 22997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:53.992542 22997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:53.996541 22997 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:33521
I20260925 12:25:53.996577 23064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:33521 every 8 connection(s)
I20260925 12:25:53.996912 22997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:53.997860 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:54.000064 23065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d: Bootstrap starting.
I20260925 12:25:54.000587 23065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:54.000818 23065 log.cc:824] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:54.001470 23065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d: No bootstrap required, opened a new log
I20260925 12:25:54.001976 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 22997
I20260925 12:25:54.002094 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:54.002278 1214 external_mini_cluster.cc:1428] Setting key b115e775db566881b639eea4be4b88a3
I20260925 12:25:54.003594 23065 raft_consensus.cc:348] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } }
I20260925 12:25:54.003775 23065 raft_consensus.cc:374] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:54.003819 23065 raft_consensus.cc:729] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d659d9393e2c4a108e9449a9ac3ec61d, State: Initialized, Role: FOLLOWER
I20260925 12:25:54.004359 23065 consensus_queue.cc:260] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } }
I20260925 12:25:54.004783 23065 raft_consensus.cc:388] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:54.004866 23065 raft_consensus.cc:482] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:54.004922 23065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:54.004966 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:54.002861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60193 (local address 127.1.47.190:33521)
0925 12:25:54.003059 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:54.003067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:54.003091 (+ 24us) server_negotiation.cc:407] Connection header received
0925 12:25:54.003577 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:54.003583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:54.003609 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:54.003720 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:54.004469 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:54.004473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:54.004473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:54.004476 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:54.004513 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:54.004524 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:54.004580 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:54.004685 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 12:25:54.004717 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260925 12:25:54.005506 23065 raft_consensus.cc:504] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } }
I20260925 12:25:54.005633 23065 leader_election.cc:302] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d659d9393e2c4a108e9449a9ac3ec61d; no voters:
I20260925 12:25:54.005800 23065 leader_election.cc:288] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:54.005841 23070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:54.005890 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:54.005978 23070 raft_consensus.cc:686] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [term 1 LEADER]: Becoming Leader. State: Replica: d659d9393e2c4a108e9449a9ac3ec61d, State: Running, Role: LEADER
I20260925 12:25:54.006089 23065 sys_catalog.cc:564] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:54.006059 23070 consensus_queue.cc:237] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } }
I20260925 12:25:54.006384 23070 sys_catalog.cc:455] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } } }
I20260925 12:25:54.006513 23070 sys_catalog.cc:458] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:54.006707 23070 sys_catalog.cc:455] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d659d9393e2c4a108e9449a9ac3ec61d. Latest consensus state: current_term: 1 leader_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d659d9393e2c4a108e9449a9ac3ec61d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 33521 } } }
I20260925 12:25:54.006843 23070 sys_catalog.cc:458] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:54.007239 23078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:54.008031 23078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:54.008780 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:54.007590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58267 (local address 127.1.47.190:33521)
0925 12:25:54.007621 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:54.007623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:54.007744 (+ 121us) server_negotiation.cc:407] Connection header received
0925 12:25:54.007810 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:54.007811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:54.007815 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:54.007894 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:54.008531 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:54.008532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:54.008532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:54.008537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:54.008553 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:54.008559 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:54.008631 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:54.008700 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 12:25:54.008713 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 12:25:54.009790 23078 catalog_manager.cc:1347] Generated new cluster ID: 403038a9f9ec44729a5bf8ef874dac56
I20260925 12:25:54.009859 23078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:54.017386 23078 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:54.017570 23078 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:54.024288 23078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d659d9393e2c4a108e9449a9ac3ec61d: Generated new TSK 0
I20260925 12:25:54.024523 23078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:54.030931 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 22997
2026-09-25T12:25:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:54 dist-test-slave-l9ww krb5kdc[23096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:54 dist-test-slave-l9ww krb5kdc[23096](info): set up 2 sockets
Sep 25 12:25:54 dist-test-slave-l9ww krb5kdc[23096](info): commencing operation
krb5kdc: starting...
W20260925 12:25:55.100472 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:55.223677 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37009
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:37009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:55.306037 23109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:55.306267 23109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:55.306290 23109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:55.307447 23109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:55.307490 23109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:55.307505 23109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:55.307518 23109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:55.307530 23109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:55.309046 23109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:37009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:37009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:55.309254 23109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:55.309422 23109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:55.314411 23114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:55.314541 23109 server_base.cc:1034] running on GCE node
W20260925 12:25:55.314419 23117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:55.314608 23115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:55.314878 23109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:55.315238 23109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:55.316399 23109 hybrid_clock.cc:648] HybridClock initialized: now 1790339155316369 us; error 64 us; skew 500 ppm
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339155, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:55.318902 23109 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:55.319363 23109 webserver.cc:460] Webserver started at http://127.1.47.190:46041/ using document root <none> and password file <none>
I20260925 12:25:55.319553 23109 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:55.319607 23109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:55.319705 23109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:55.320785 23109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "3cc8744d19cc4210ac314d42acc4f41d"
format_stamp: "Formatted at 2026-09-25 12:25:55 on dist-test-slave-l9ww"
server_key: "71dbd84a9a21a8f3abdebff2d62ce5f1"
server_key_iv: "8dc1c70629e07e9f83b04183d28c7f3e"
server_key_version: "encryptionkey@0"
I20260925 12:25:55.321084 23109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "3cc8744d19cc4210ac314d42acc4f41d"
format_stamp: "Formatted at 2026-09-25 12:25:55 on dist-test-slave-l9ww"
server_key: "71dbd84a9a21a8f3abdebff2d62ce5f1"
server_key_iv: "8dc1c70629e07e9f83b04183d28c7f3e"
server_key_version: "encryptionkey@0"
I20260925 12:25:55.322358 23109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:55.323058 23124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:55.323279 23109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:55.323370 23109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "3cc8744d19cc4210ac314d42acc4f41d"
format_stamp: "Formatted at 2026-09-25 12:25:55 on dist-test-slave-l9ww"
server_key: "71dbd84a9a21a8f3abdebff2d62ce5f1"
server_key_iv: "8dc1c70629e07e9f83b04183d28c7f3e"
server_key_version: "encryptionkey@0"
I20260925 12:25:55.323428 23109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:55.350396 23109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:55.352553 23109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:55.352672 23109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:55.356559 23109 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:37009
I20260925 12:25:55.356637 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:37009 every 8 connection(s)
I20260925 12:25:55.356962 23109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:55.357894 23177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:55.358695 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23109
I20260925 12:25:55.358788 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:55.358970 1214 external_mini_cluster.cc:1428] Setting key 5bf1f260b00b82d981f495d8fc06cfdb
I20260925 12:25:55.361088 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d: Bootstrap starting.
I20260925 12:25:55.361795 23177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:55.362099 23177 log.cc:824] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:55.362818 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d: No bootstrap required, opened a new log
I20260925 12:25:55.365020 23177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } }
I20260925 12:25:55.365161 23177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:55.365183 23177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc8744d19cc4210ac314d42acc4f41d, State: Initialized, Role: FOLLOWER
I20260925 12:25:55.365350 23177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } }
I20260925 12:25:55.365414 23177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:55.365437 23177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:55.365509 23177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 0 FOLLOWER]: Advancing to term 1
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:55.366112 23177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } }
I20260925 12:25:55.366236 23177 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc8744d19cc4210ac314d42acc4f41d; no voters:
I20260925 12:25:55.366430 23177 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:55.366464 23182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:55.366581 23182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [term 1 LEADER]: Becoming Leader. State: Replica: 3cc8744d19cc4210ac314d42acc4f41d, State: Running, Role: LEADER
I20260925 12:25:55.366693 23182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } }
I20260925 12:25:55.366696 23177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:55.367136 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cc8744d19cc4210ac314d42acc4f41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } } }
I20260925 12:25:55.367273 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:55.367125 23182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc8744d19cc4210ac314d42acc4f41d. Latest consensus state: current_term: 1 leader_uuid: "3cc8744d19cc4210ac314d42acc4f41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc8744d19cc4210ac314d42acc4f41d" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 37009 } } }
I20260925 12:25:55.367550 23182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:55.367761 23187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:55.368372 23187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:55.370108 23187 catalog_manager.cc:1347] Generated new cluster ID: 580f3c18648f4df8a82ccc66465a6748
I20260925 12:25:55.370155 23187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:55.370674 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:55.359516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39787 (local address 127.1.47.190:37009)
0925 12:25:55.359714 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:55.359723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:55.359755 (+ 32us) server_negotiation.cc:407] Connection header received
0925 12:25:55.360311 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:55.360320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:55.360386 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:55.360527 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:55.361099 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:55.362462 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:55.363294 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:55.363968 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:55.364085 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:55.366628 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:55.366633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:55.366633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:55.366648 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:55.368870 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:55.369380 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:55.369383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:55.369384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:55.369523 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:55.369791 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:55.369792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:55.369793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:55.370095 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:55.370224 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:55.370347 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 12:25:55.370502 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260925 12:25:55.372798 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:55.382294 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:55.374137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33177 (local address 127.1.47.190:37009)
0925 12:25:55.374182 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:55.374183 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:55.376166 (+ 1983us) server_negotiation.cc:407] Connection header received
0925 12:25:55.376236 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:55.376237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:55.376248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:55.376297 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:55.376849 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:55.377360 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:55.378084 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:55.378322 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:55.378399 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:55.379627 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:55.379629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:55.379629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:55.379635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:55.381058 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:55.381461 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:55.381462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:55.381463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:55.381522 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:55.381732 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:55.381733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:55.381733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:55.381846 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:55.381959 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:55.382094 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 12:25:55.382183 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:25:55.392223 23187 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:55.392784 23187 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:55.402661 23187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cc8744d19cc4210ac314d42acc4f41d: Generated new TSK 0
I20260925 12:25:55.402894 23187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:55.413657 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23109
2026-09-25T12:25:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23208](info): set up 2 sockets
Sep 25 12:25:55 dist-test-slave-l9ww krb5kdc[23208](info): commencing operation
krb5kdc: starting...
W20260925 12:25:56.476708 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:56 dist-test-slave-l9ww krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:56.598897 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41423
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:56.679391 23221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:56.679672 23221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:56.679713 23221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:56.680918 23221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:56.680999 23221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:56.681025 23221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:56.681047 23221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:56.681074 23221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:56.682602 23221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:56.682901 23221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:56.683136 23221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:56.688026 23226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:56.688019 23227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:56.688019 23229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:56.688099 23221 server_base.cc:1034] running on GCE node
I20260925 12:25:56.688439 23221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:56.688686 23221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:56.689845 23221 hybrid_clock.cc:648] HybridClock initialized: now 1790339156689815 us; error 43 us; skew 500 ppm
Sep 25 12:25:56 dist-test-slave-l9ww krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:56.693037 23221 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:56.693624 23221 webserver.cc:460] Webserver started at http://127.1.47.190:38393/ using document root <none> and password file <none>
I20260925 12:25:56.693902 23221 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:56.693954 23221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:56.694114 23221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:56.695295 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "a07b652bcb224ada947cd6f2a4fdbc04"
format_stamp: "Formatted at 2026-09-25 12:25:56 on dist-test-slave-l9ww"
server_key: "8258a17ed1193826903ff1494b86f82c"
server_key_iv: "40e47ac4eb354b1efeaf79a5990c5ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:56.695689 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "a07b652bcb224ada947cd6f2a4fdbc04"
format_stamp: "Formatted at 2026-09-25 12:25:56 on dist-test-slave-l9ww"
server_key: "8258a17ed1193826903ff1494b86f82c"
server_key_iv: "40e47ac4eb354b1efeaf79a5990c5ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:56.697209 23221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:56.698105 23236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:56.698414 23221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:56.698529 23221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "a07b652bcb224ada947cd6f2a4fdbc04"
format_stamp: "Formatted at 2026-09-25 12:25:56 on dist-test-slave-l9ww"
server_key: "8258a17ed1193826903ff1494b86f82c"
server_key_iv: "40e47ac4eb354b1efeaf79a5990c5ba7"
server_key_version: "encryptionkey@0"
I20260925 12:25:56.698606 23221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:56.713821 23221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:56.716343 23221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:56.716521 23221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:56.720645 23221 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:41423
I20260925 12:25:56.720686 23288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:41423 every 8 connection(s)
I20260925 12:25:56.721048 23221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:56.721900 23289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:56.723760 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23221
I20260925 12:25:56.723914 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:56.724088 1214 external_mini_cluster.cc:1428] Setting key a8728b54fb33120cba15db6361acd206
I20260925 12:25:56.724371 23289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04: Bootstrap starting.
I20260925 12:25:56.725143 23289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:56.725479 23289 log.cc:824] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:56.726310 23289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04: No bootstrap required, opened a new log
I20260925 12:25:56.728631 23289 raft_consensus.cc:348] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } }
I20260925 12:25:56.728814 23289 raft_consensus.cc:374] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:56.728850 23289 raft_consensus.cc:729] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07b652bcb224ada947cd6f2a4fdbc04, State: Initialized, Role: FOLLOWER
I20260925 12:25:56.729017 23289 consensus_queue.cc:260] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } }
I20260925 12:25:56.729106 23289 raft_consensus.cc:388] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:56.729144 23289 raft_consensus.cc:482] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:56.729193 23289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:56.729809 23289 raft_consensus.cc:504] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } }
I20260925 12:25:56.729930 23289 leader_election.cc:302] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a07b652bcb224ada947cd6f2a4fdbc04; no voters:
I20260925 12:25:56.730108 23289 leader_election.cc:288] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:56.730299 23295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:56.730387 23289 sys_catalog.cc:564] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:56.730533 23295 raft_consensus.cc:686] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [term 1 LEADER]: Becoming Leader. State: Replica: a07b652bcb224ada947cd6f2a4fdbc04, State: Running, Role: LEADER
I20260925 12:25:56.730793 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:56.724649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34571 (local address 127.1.47.190:41423)
0925 12:25:56.724867 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:56.724877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:56.724913 (+ 36us) server_negotiation.cc:407] Connection header received
0925 12:25:56.725572 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:56.725582 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:56.725650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:56.725799 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:56.726036 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:56.727424 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:56.728266 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:56.728912 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:56.729052 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:56.729752 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:56.729756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:56.729757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:56.729976 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:56.730113 (+ 137us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:56.730132 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:56.730203 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:56.730515 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 12:25:56.730584 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260925 12:25:56.730759 23295 consensus_queue.cc:237] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } }
I20260925 12:25:56.731379 23294 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } } }
I20260925 12:25:56.731371 23295 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a07b652bcb224ada947cd6f2a4fdbc04. Latest consensus state: current_term: 1 leader_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07b652bcb224ada947cd6f2a4fdbc04" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41423 } } }
I20260925 12:25:56.731468 23295 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:56.731642 23294 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04 [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:56.731705 23306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:56.732390 23306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:56.732450 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:56.733986 23306 catalog_manager.cc:1347] Generated new cluster ID: d1543ab0af364fb99e6a4c413ba442e5
I20260925 12:25:56.734045 23306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:56.742640 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:56.733995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51573 (local address 127.1.47.190:41423)
0925 12:25:56.734032 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:56.734034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:56.739169 (+ 5135us) server_negotiation.cc:407] Connection header received
0925 12:25:56.739234 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:56.739235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:56.739246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:56.739305 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:56.739471 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:56.740234 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:56.741263 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:56.741552 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:56.741610 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:56.742360 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:56.742361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:56.742361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:56.742366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:56.742381 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:56.742385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:56.742433 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:56.742511 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 12:25:56.742533 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":18}
I20260925 12:25:56.746579 23306 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:56.747282 23306 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:56.761902 23306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a07b652bcb224ada947cd6f2a4fdbc04: Generated new TSK 0
I20260925 12:25:56.762146 23306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:56.765305 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23221
2026-09-25T12:25:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:56 dist-test-slave-l9ww krb5kdc[23320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:56 dist-test-slave-l9ww krb5kdc[23320](info): set up 2 sockets
Sep 25 12:25:56 dist-test-slave-l9ww krb5kdc[23320](info): commencing operation
krb5kdc: starting...
W20260925 12:25:57.860795 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:57 dist-test-slave-l9ww krb5kdc[23320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:57.991499 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45839
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:45839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:58.073050 23333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:58.073323 23333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:58.073364 23333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:58.074570 23333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:58.074647 23333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:58.074673 23333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:58.074707 23333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:58.074729 23333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:58.076376 23333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:45839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:45839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:58.076669 23333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:58.076911 23333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:58.081836 23341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:58.081849 23338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:58.081924 23333 server_base.cc:1034] running on GCE node
W20260925 12:25:58.081851 23339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:58.082331 23333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:58.082616 23333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:58.083770 23333 hybrid_clock.cc:648] HybridClock initialized: now 1790339158083755 us; error 30 us; skew 500 ppm
Sep 25 12:25:58 dist-test-slave-l9ww krb5kdc[23320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339158, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:58.093856 23333 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:58.094326 23333 webserver.cc:460] Webserver started at http://127.1.47.190:46067/ using document root <none> and password file <none>
I20260925 12:25:58.094560 23333 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:58.094628 23333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:58.094789 23333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:58.095921 23333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "11e0089dd5fd41f09ce2c5e29e38f4df"
format_stamp: "Formatted at 2026-09-25 12:25:58 on dist-test-slave-l9ww"
server_key: "1089766c720f4eacc8d7267ac536001d"
server_key_iv: "a32e7fc3d2b6e0e0c988793db70d01ed"
server_key_version: "encryptionkey@0"
I20260925 12:25:58.096333 23333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "11e0089dd5fd41f09ce2c5e29e38f4df"
format_stamp: "Formatted at 2026-09-25 12:25:58 on dist-test-slave-l9ww"
server_key: "1089766c720f4eacc8d7267ac536001d"
server_key_iv: "a32e7fc3d2b6e0e0c988793db70d01ed"
server_key_version: "encryptionkey@0"
I20260925 12:25:58.097661 23333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:25:58.098387 23348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:58.098557 23333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:25:58.098664 23333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "11e0089dd5fd41f09ce2c5e29e38f4df"
format_stamp: "Formatted at 2026-09-25 12:25:58 on dist-test-slave-l9ww"
server_key: "1089766c720f4eacc8d7267ac536001d"
server_key_iv: "a32e7fc3d2b6e0e0c988793db70d01ed"
server_key_version: "encryptionkey@0"
I20260925 12:25:58.098737 23333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:58.102177 23333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:58.104264 23333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:58.104420 23333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:58.108481 23333 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:45839
I20260925 12:25:58.108520 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:45839 every 8 connection(s)
I20260925 12:25:58.108865 23333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:58.109850 23401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:58.112162 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df: Bootstrap starting.
I20260925 12:25:58.112763 23401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:58.113054 23401 log.cc:824] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:58.113751 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df: No bootstrap required, opened a new log
I20260925 12:25:58.115324 23401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } }
I20260925 12:25:58.115463 23401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 12:25:58.115486 23401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e0089dd5fd41f09ce2c5e29e38f4df, State: Initialized, Role: FOLLOWER
I20260925 12:25:58.115640 23401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } }
I20260925 12:25:58.115710 23401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 12:25:58.115731 23401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 12:25:58.115759 23401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:58.116495 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23333
I20260925 12:25:58.116631 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:58.116739 23401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } }
I20260925 12:25:58.116845 1214 external_mini_cluster.cc:1428] Setting key 3aa35c4658256486e2fd0c50ef1c2a37
I20260925 12:25:58.116859 23401 leader_election.cc:302] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11e0089dd5fd41f09ce2c5e29e38f4df; no voters:
I20260925 12:25:58.117051 23401 leader_election.cc:288] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 12:25:58.117072 23404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:58.117204 23404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [term 1 LEADER]: Becoming Leader. State: Replica: 11e0089dd5fd41f09ce2c5e29e38f4df, State: Running, Role: LEADER
I20260925 12:25:58.117340 23401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:58.117342 23404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } }
I20260925 12:25:58.117767 23406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } } }
I20260925 12:25:58.117873 23405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e0089dd5fd41f09ce2c5e29e38f4df. Latest consensus state: current_term: 1 leader_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0089dd5fd41f09ce2c5e29e38f4df" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 45839 } } }
I20260925 12:25:58.117981 23405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:58.117906 23406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:58.118454 23415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:58.118988 23415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:58.120035 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:58.117552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44979 (local address 127.1.47.190:45839)
0925 12:25:58.118180 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:58.118190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:58.118227 (+ 37us) server_negotiation.cc:407] Connection header received
0925 12:25:58.118389 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:58.118396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:58.118429 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:58.118624 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:58.119377 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:58.119380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:58.119381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:58.119385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:58.119428 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:58.119443 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:58.119497 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:58.119779 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 12:25:58.119823 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":70,"threads_started":1}
I20260925 12:25:58.120661 23415 catalog_manager.cc:1347] Generated new cluster ID: f13afee97f7341b094cfd493b48b36a8
I20260925 12:25:58.120711 23415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:58.121029 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:58.123849 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:58.122582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35429 (local address 127.1.47.190:45839)
0925 12:25:58.122617 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:58.122619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:58.122879 (+ 260us) server_negotiation.cc:407] Connection header received
0925 12:25:58.122913 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:58.122914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:58.122920 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:58.122982 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:58.123625 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:58.123627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:58.123627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 12:25:58.123631 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:58.123646 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 12:25:58.123651 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:58.123712 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:58.123770 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 12:25:58.123785 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 12:25:58.126899 23415 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:58.127079 23415 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:58.140458 23415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11e0089dd5fd41f09ce2c5e29e38f4df: Generated new TSK 0
I20260925 12:25:58.140707 23415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:58.150892 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23333
2026-09-25T12:25:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1385 ms)
[----------] 18 tests from AuthTokenIssuingTest (24200 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:25:58 dist-test-slave-l9ww krb5kdc[23432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:25:58 dist-test-slave-l9ww krb5kdc[23432](info): set up 2 sockets
Sep 25 12:25:58 dist-test-slave-l9ww krb5kdc[23432](info): commencing operation
krb5kdc: starting...
W20260925 12:25:59.196171 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:25:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:25:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:25:59.320683 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41599
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:25:59.405932 23445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:25:59.406148 23445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:59.406165 23445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:59.406186 23445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:59.407267 23445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:59.407307 23445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:59.407322 23445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:59.407335 23445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:59.407347 23445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:59.408910 23445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:59.409118 23445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:59.409300 23445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:59.414090 23451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.414113 23450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.414090 23453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:59.414440 23445 server_base.cc:1034] running on GCE node
I20260925 12:25:59.414685 23445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:59.415051 23445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:59.416214 23445 hybrid_clock.cc:648] HybridClock initialized: now 1790339159416193 us; error 37 us; skew 500 ppm
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:59.418735 23445 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:25:59.419159 23445 webserver.cc:460] Webserver started at http://127.1.47.190:41289/ using document root <none> and password file <none>
I20260925 12:25:59.419410 23445 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:59.419479 23445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:59.419622 23445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:59.420770 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "4a44fa1647254ebcba133e2220dd1eaa"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "853c7c10ee0857ada1f8da7692dda6fe"
server_key_iv: "10085a4af99a49cdb4ba3a53904d9168"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.421129 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "4a44fa1647254ebcba133e2220dd1eaa"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "853c7c10ee0857ada1f8da7692dda6fe"
server_key_iv: "10085a4af99a49cdb4ba3a53904d9168"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.422420 23445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:25:59.423105 23460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:59.423295 23445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 12:25:59.423391 23445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "4a44fa1647254ebcba133e2220dd1eaa"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "853c7c10ee0857ada1f8da7692dda6fe"
server_key_iv: "10085a4af99a49cdb4ba3a53904d9168"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.423472 23445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:59.449198 23445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:59.451095 23445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:59.451272 23445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:59.455940 23445 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:41599
I20260925 12:25:59.455991 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:41599 every 8 connection(s)
I20260925 12:25:59.456378 23445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:25:59.457448 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:59.459020 23513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:59.466033 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23445
I20260925 12:25:59.466158 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:25:59.466363 1214 external_mini_cluster.cc:1428] Setting key af16563ac4227d878bd2f05cb8f78cd4
I20260925 12:25:59.469282 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.459759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41599 (local address 127.0.0.1:55090)
0925 12:25:59.459981 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.460026 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.460493 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.460817 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.460820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.461079 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.461237 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.461241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.462252 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.462253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.462985 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.462987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.463548 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.463549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.463617 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.464210 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.464218 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.465629 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.467620 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.467626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.467628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.467825 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.468133 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.468133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.468134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.468220 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.468662 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.468667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.468732 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.468837 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.469085 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":99,"thread_start_us":143,"threads_started":1}
I20260925 12:25:59.469282 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.459885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.1.47.190:41599)
0925 12:25:59.460057 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.460061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.460105 (+ 44us) server_negotiation.cc:407] Connection header received
0925 12:25:59.460656 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.460658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.460677 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.460772 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.461329 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.462146 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.463074 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.463407 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.463512 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.465761 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.465768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.465769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.465775 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.467508 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.467965 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.467968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.467968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.468053 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.468312 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.468313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.468313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.468582 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.468672 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.468882 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.469020 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260925 12:25:59.470417 23513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:25:59.470863 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.470707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39232)
0925 12:25:59.470769 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.470826 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 12:25:59.471055 23462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
I20260925 12:25:59.471647 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.471527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39238)
0925 12:25:59.471580 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.471614 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:25:59.471882 23513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:37911: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
I20260925 12:25:59.490929 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:37911
--webserver_interface=127.1.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:25:59.521145 23513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } attempt: 1
I20260925 12:25:59.521740 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.521589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39250)
0925 12:25:59.521653 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.521702 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:25:59.522125 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.522035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39266)
0925 12:25:59.522076 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.522103 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 12:25:59.522228 23513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:37911: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
W20260925 12:25:59.573732 23520 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:25:59.573968 23520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:59.573985 23520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:59.574007 23520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:59.575220 23520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:59.575302 23520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:59.575328 23520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:59.575352 23520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:59.575371 23520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:59.576913 23520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:59.577176 23520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:59.577428 23520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:59.582384 23528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.582381 23525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.582410 23526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:59.582671 23520 server_base.cc:1034] running on GCE node
I20260925 12:25:59.582918 23520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:59.583184 23520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:59.584378 23520 hybrid_clock.cc:648] HybridClock initialized: now 1790339159584363 us; error 30 us; skew 500 ppm
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:59.586980 23520 init.cc:377] Logged in from keytab as kudu/127.1.47.189@KRBTEST.COM (short username kudu)
I20260925 12:25:59.587478 23520 webserver.cc:460] Webserver started at http://127.1.47.189:44915/ using document root <none> and password file <none>
I20260925 12:25:59.587719 23520 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:59.587766 23520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:59.587932 23520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:59.589057 23520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data/instance:
uuid: "0263740cad6d42ed95992087cb050f50"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "3f643714dd266dc3bedc29335aeee5ec"
server_key_iv: "8e43abd2233487d209a43d773c01f04e"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.589434 23520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal/instance:
uuid: "0263740cad6d42ed95992087cb050f50"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "3f643714dd266dc3bedc29335aeee5ec"
server_key_iv: "8e43abd2233487d209a43d773c01f04e"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.590379 23513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } attempt: 2
I20260925 12:25:59.590888 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.590721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39268)
0925 12:25:59.590789 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.590826 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 12:25:59.590986 23520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:25:59.591406 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.591287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39280)
0925 12:25:59.591344 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.591379 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 12:25:59.591593 23513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:37911: Network error: Client connection negotiation failed: client connection to 127.1.47.189:37911: connect: Connection refused (error 111)
I20260925 12:25:59.591984 23535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:59.592171 23520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:59.592288 23520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "0263740cad6d42ed95992087cb050f50"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "3f643714dd266dc3bedc29335aeee5ec"
server_key_iv: "8e43abd2233487d209a43d773c01f04e"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.592376 23520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:59.616194 23520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:59.618424 23520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:59.618600 23520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:59.622905 23520 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.189:37911
I20260925 12:25:59.622958 23587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.189:37911 every 8 connection(s)
I20260925 12:25:59.623378 23520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
I20260925 12:25:59.624492 23588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:59.626128 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23520
I20260925 12:25:59.626255 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-1/wal/instance
I20260925 12:25:59.626160 23588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:25:59.626420 1214 external_mini_cluster.cc:1428] Setting key 154e1d3ef70c47e994f6031970c4cfc6
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.188@KRBTEST.COM; defaulting to no policy
I20260925 12:25:59.635910 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.627027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.1.47.190:41599)
0925 12:25:59.627071 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.627073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.627319 (+ 246us) server_negotiation.cc:407] Connection header received
0925 12:25:59.627916 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.627918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.627934 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.627998 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.628635 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.629144 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.630222 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.630452 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.630540 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.632817 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.632819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.632819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.632834 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.634304 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.634822 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.634823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.634823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.634888 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.635155 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.635156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.635157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.635283 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.635360 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.635694 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.635786 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:25:59.636103 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.627023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41599 (local address 127.0.0.1:55102)
0925 12:25:59.627189 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.627227 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.627791 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.628075 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.628078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.628343 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.628537 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.628541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.629243 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.629244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.630118 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.630120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.630575 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.630577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.630692 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.631303 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.631311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.632703 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.634408 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.634416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.634419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.634717 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.634956 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.634957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.634957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.635079 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.635351 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.635358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.635551 (+ 193us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.635670 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.635835 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.188@KRBTEST.COM" created.
I20260925 12:25:59.636680 23588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:25:59.644366 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.636974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39288)
0925 12:25:59.637030 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.637050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.637097 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.637349 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.637350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.637520 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.637585 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.637587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.638459 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.638460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.638970 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.638971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.639475 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.639476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.639539 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.640010 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.640015 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.641442 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.643255 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.643256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.643256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.643423 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.643659 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.643659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.643660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.643713 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.644049 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.644050 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.644089 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.644154 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.644258 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 12:25:59.644430 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.637020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.1.47.189:37911)
0925 12:25:59.637214 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.637222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.637238 (+ 16us) server_negotiation.cc:407] Connection header received
0925 12:25:59.637257 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.637258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.637270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.637330 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.637660 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.638348 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.639062 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.639390 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.639467 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.641547 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.641555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.641556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.641562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.643124 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.643516 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.643519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.643520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.643578 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.643785 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.643786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.643786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.643979 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.644072 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.644183 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.644308 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
I20260925 12:25:59.645056 23588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:25:59.645411 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.645275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59166)
0925 12:25:59.645342 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.645386 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 12:25:59.645555 23539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
I20260925 12:25:59.646004 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.645905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59170)
0925 12:25:59.645958 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.645982 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 12:25:59.646142 23588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:33145: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
I20260925 12:25:59.651306 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:33145
--webserver_interface=127.1.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:25:59.682780 23513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } attempt: 3
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:25:59.691004 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.683311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39294)
0925 12:25:59.683382 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.683392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.683456 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.683631 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.683632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.683803 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.683920 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.683926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.684622 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.684623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.685163 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.685165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.685598 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.685599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.685686 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.686198 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.686204 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.687806 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.689718 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.689719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.689720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.689927 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.690209 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.690210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.690210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.690287 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.690606 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.690608 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.690663 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.690767 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.690884 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 12:25:59.691004 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.683376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.1.47.189:37911)
0925 12:25:59.683441 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.683443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.683455 (+ 12us) server_negotiation.cc:407] Connection header received
0925 12:25:59.683520 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.683521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.683535 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.683622 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.684006 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.684527 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.685255 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.685483 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.685599 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.687964 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.687966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.687966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.687978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.689562 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.690040 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.690042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.690042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.690111 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.690378 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.690379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.690380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.690520 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.690615 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.690795 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.690884 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 12:25:59.691520 23513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:25:59.691985 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.691838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59178)
0925 12:25:59.691925 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.691956 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 12:25:59.692406 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.692322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59184)
0925 12:25:59.692357 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.692389 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260925 12:25:59.692509 23513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:33145: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
I20260925 12:25:59.695372 23588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } attempt: 1
I20260925 12:25:59.695861 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.695717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59200)
0925 12:25:59.695772 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.695802 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:25:59.696252 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.696140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59208)
0925 12:25:59.696187 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.696228 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 12:25:59.696631 23588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:33145: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
I20260925 12:25:59.723682 23513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } attempt: 1
I20260925 12:25:59.724282 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.724085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59216)
0925 12:25:59.724203 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.724249 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260925 12:25:59.724728 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.724638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59224)
0925 12:25:59.724683 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.724710 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:25:59.724864 23513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:33145: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
W20260925 12:25:59.735637 23595 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:25:59.735904 23595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:25:59.735936 23595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:25:59.735958 23595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:25:59.737142 23595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:25:59.737200 23595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:25:59.737222 23595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:25:59.737259 23595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:25:59.737285 23595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:25:59.738749 23595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:41599,127.1.47.189:37911,127.1.47.188:33145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:33145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:25:59.739010 23595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:25:59.739250 23595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:25:59.744107 23601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.744114 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:25:59.744297 23603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:25:59.744190 23595 server_base.cc:1034] running on GCE node
I20260925 12:25:59.744527 23595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:25:59.744871 23595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:25:59.746018 23595 hybrid_clock.cc:648] HybridClock initialized: now 1790339159746008 us; error 28 us; skew 500 ppm
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:25:59.748651 23595 init.cc:377] Logged in from keytab as kudu/127.1.47.188@KRBTEST.COM (short username kudu)
I20260925 12:25:59.749153 23595 webserver.cc:460] Webserver started at http://127.1.47.188:45573/ using document root <none> and password file <none>
I20260925 12:25:59.749354 23595 fs_manager.cc:359] Metadata directory not provided
I20260925 12:25:59.749395 23595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:25:59.749486 23595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:25:59.750509 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data/instance:
uuid: "268d928d936e47f5a79aca807383b5e7"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "57fc2ccbf3e0a48cb3fd300969b3bf57"
server_key_iv: "d2f06fec17d13fa0830fd81f3dc042d9"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.750815 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal/instance:
uuid: "268d928d936e47f5a79aca807383b5e7"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "57fc2ccbf3e0a48cb3fd300969b3bf57"
server_key_iv: "d2f06fec17d13fa0830fd81f3dc042d9"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.752149 23595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 12:25:59.752892 23610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:25:59.753082 23595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:25:59.753186 23595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "268d928d936e47f5a79aca807383b5e7"
format_stamp: "Formatted at 2026-09-25 12:25:59 on dist-test-slave-l9ww"
server_key: "57fc2ccbf3e0a48cb3fd300969b3bf57"
server_key_iv: "d2f06fec17d13fa0830fd81f3dc042d9"
server_key_version: "encryptionkey@0"
I20260925 12:25:59.753276 23595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:25:59.764864 23588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } attempt: 2
I20260925 12:25:59.765360 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.765207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59230)
0925 12:25:59.765279 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.765322 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 12:25:59.765800 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.765694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59242)
0925 12:25:59.765746 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.765778 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 12:25:59.765914 23588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:33145: Network error: Client connection negotiation failed: client connection to 127.1.47.188:33145: connect: Connection refused (error 111)
I20260925 12:25:59.766391 23595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:25:59.768841 23595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:25:59.769009 23595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:25:59.773279 23595 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.188:33145
I20260925 12:25:59.773326 23662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.188:33145 every 8 connection(s)
I20260925 12:25:59.773718 23595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
I20260925 12:25:59.774731 23663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:25:59.775986 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23595
I20260925 12:25:59.776098 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790338888974512-1214-0/minicluster-data/master-2/wal/instance
I20260925 12:25:59.776258 1214 external_mini_cluster.cc:1428] Setting key 7dd606e1d9ca8ea699d71a234399957d
I20260925 12:25:59.776489 23663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:25:59.786094 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.777348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55106 (local address 127.1.47.190:41599)
0925 12:25:59.777401 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.777403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.777511 (+ 108us) server_negotiation.cc:407] Connection header received
0925 12:25:59.777851 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.777852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.777864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.777930 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.778505 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.779076 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.780124 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.780370 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.780423 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.782857 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.782858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.782859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.782868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.784554 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.785087 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.785088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.785088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.785138 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.785424 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.785425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.785425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.785552 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.785623 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.785915 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.785996 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:25:59.786329 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.777256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:41599 (local address 127.0.0.1:55106)
0925 12:25:59.777396 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.777439 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.777731 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.777961 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.777963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.778233 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.778390 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.778394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.779159 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.779160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.780048 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.780051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.780463 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.780464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.780573 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.781123 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.781131 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.782761 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.784635 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.784640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.784642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.785010 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.785230 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.785231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.785231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.785347 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.785627 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.785634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.785780 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.785893 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.786050 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260925 12:25:59.786329 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.776822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.1.47.188:33145)
0925 12:25:59.777013 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.777023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.777059 (+ 36us) server_negotiation.cc:407] Connection header received
0925 12:25:59.777731 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.777740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.777765 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.777852 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.778478 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.779407 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.780241 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.780602 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.780672 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.783308 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.783317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.783317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.783324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.785134 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.785471 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.785475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.785475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.785532 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.785739 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.785740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.785740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.785886 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.786028 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.786107 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.786187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260925 12:25:59.786981 23663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:25:59.794332 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.787320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.1.47.189:37911)
0925 12:25:59.787363 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.787364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.787373 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:25:59.787397 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.787398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.787408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.787471 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.787710 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.788301 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.788985 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.789247 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.789328 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.791333 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.791335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.791335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.791342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.793100 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.793507 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.793508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.793509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.793596 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.793849 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.793850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.793850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.793971 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.794030 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.794196 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.794262 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:59.794409 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.787263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:37911 (local address 127.0.0.1:39306)
0925 12:25:59.787299 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.787310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.787342 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.787479 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.787479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.787593 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.787633 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.787634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.788373 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.788374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.788904 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.788906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.789340 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.789341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.789416 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.789844 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.789848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.791232 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.793176 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.793177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.793178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.793406 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.793678 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.793679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.793679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.793768 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.794043 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.794044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.794092 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.794173 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.794286 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 12:25:59.794910 23663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:25:59.795720 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.787149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.1.47.190:41599)
0925 12:25:59.787181 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.787183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.787193 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:25:59.787263 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.787264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.787273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.787329 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.787914 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.788390 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.789065 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.789239 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.789289 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.792652 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.792653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.792653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.792660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.794467 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.794917 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.794918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.794918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.794974 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.795218 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.795219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.795219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.795369 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.795519 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.795582 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.795646 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:25:59.800012 23513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } attempt: 2
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:25:59.803745 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.795233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59262)
0925 12:25:59.795272 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.795282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.795318 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.795754 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.795755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.795926 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.795964 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.795965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.796794 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.796795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.797544 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.797547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.797917 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.797918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.797976 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.798448 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.798454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.800004 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.802080 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.802081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.802082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.802300 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.803189 (+ 889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.803190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.803191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.803246 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.803497 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.803498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.803532 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.803579 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.803673 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 12:25:59.804054 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.795289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.1.47.188:33145)
0925 12:25:59.795322 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.795324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.795331 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:25:59.795369 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.795370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.795378 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.795438 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.796030 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.796714 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.797629 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.797826 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.797902 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.800105 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.800106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.800107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.800114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.802000 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.803049 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.803050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.803050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.803111 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.803318 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.803319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.803319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.803433 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.803864 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.803932 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.804004 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:25:59.804873 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.796427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.1.47.189:37911)
0925 12:25:59.796474 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.796476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.796514 (+ 38us) server_negotiation.cc:407] Connection header received
0925 12:25:59.796581 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.796582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.796591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.796635 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.797133 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.797824 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.798466 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.798818 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.798895 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.801357 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.801359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.801359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.801366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.802948 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.803344 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.803345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.803346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.803403 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.803644 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.803645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.803645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.804071 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.804260 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.804681 (+ 421us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.804762 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:25:59.805665 23663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: Bootstrap starting.
I20260925 12:25:59.806438 23663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:59.806809 23663 log.cc:824] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:59.807190 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:25:59.807931 23663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: No bootstrap required, opened a new log
I20260925 12:25:59.808367 23487 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4a44fa1647254ebcba133e2220dd1eaa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55108
I20260925 12:25:59.809584 23663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.809756 23663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 268d928d936e47f5a79aca807383b5e7, State: Initialized, Role: FOLLOWER
I20260925 12:25:59.809856 23663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.810053 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.800318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59268)
0925 12:25:59.800375 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.800386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.800425 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.800666 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.800668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.800872 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.800924 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.800926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.801876 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.801877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.802641 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.802644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.803596 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.803597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.803683 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.804300 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.804305 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.805761 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.807770 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.807771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.807772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.808079 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.809329 (+ 1250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.809330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.809331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.809395 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.809694 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.809695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.809743 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.809828 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.809929 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 12:25:59.810190 23663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:59.810190 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.810297 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.810431 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.800385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.1.47.188:33145)
0925 12:25:59.800542 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.800545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.800552 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:25:59.800573 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.800573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.800585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.800653 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.801021 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.801801 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.802725 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.803520 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.803588 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.805852 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.805854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.805854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.805861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.807671 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.809168 (+ 1497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.809169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.809175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.809252 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.809470 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.809471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.809471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.809614 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.809688 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.810183 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.810268 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260925 12:25:59.811990 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: Bootstrap starting.
W20260925 12:25:59.812332 23681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:25:59.812384 23681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:25:59.812829 23513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:59.813205 23513 log.cc:824] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:59.814041 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: No bootstrap required, opened a new log
I20260925 12:25:59.815420 23513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.815560 23513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a44fa1647254ebcba133e2220dd1eaa, State: Initialized, Role: FOLLOWER
I20260925 12:25:59.815634 23513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.816023 23513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:59.815994 23684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.816078 23684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.813868 23595 master_runner.cc:186] Error getting master registration for 127.1.47.190:41599: OK, instance_id { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" instance_seqno: 1790339159454608 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:25:59.816399 23595 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 12:25:59.816653 23487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 12:25:59.816727 23487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:59.817602 23487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.817871 23487 leader_election.cc:288] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [CANDIDATE]: Term 1 election: Requested vote from peers 0263740cad6d42ed95992087cb050f50 (127.1.47.189:37911), 268d928d936e47f5a79aca807383b5e7 (127.1.47.188:33145)
W20260925 12:25:59.818406 23695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:25:59.818460 23695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:25:59.818398 23637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a44fa1647254ebcba133e2220dd1eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "268d928d936e47f5a79aca807383b5e7"
I20260925 12:25:59.818511 23637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:59.820155 23637 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a44fa1647254ebcba133e2220dd1eaa in term 1.
I20260925 12:25:59.824424 23464 leader_election.cc:302] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 268d928d936e47f5a79aca807383b5e7, 4a44fa1647254ebcba133e2220dd1eaa; no voters:
I20260925 12:25:59.824585 23684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:25:59.824754 23684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [term 1 LEADER]: Becoming Leader. State: Replica: 4a44fa1647254ebcba133e2220dd1eaa, State: Running, Role: LEADER
I20260925 12:25:59.824860 23684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.825331 23684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a44fa1647254ebcba133e2220dd1eaa. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.825412 23684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:59.826776 23705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:25:59.824775 23445 master_runner.cc:186] Error getting master registration for 127.1.47.189:37911: OK, instance_id { permanent_uuid: "0263740cad6d42ed95992087cb050f50" instance_seqno: 1790339159621746 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:25:59.827350 23445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 12:25:59.828914 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.819486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.1.47.190:41599)
0925 12:25:59.819527 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.819528 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.819667 (+ 139us) server_negotiation.cc:407] Connection header received
0925 12:25:59.819751 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.819752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.819762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.819827 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.820604 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.821076 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.822112 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.822409 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.822470 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.825757 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.825759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.825759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.825767 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.827563 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.828068 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.828069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.828070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.828137 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.828339 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.828340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.828340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.828468 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.828547 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.828743 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.828821 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 12:25:59.838204 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.819263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.1.47.188:33145)
0925 12:25:59.819311 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.819314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.827828 (+ 8514us) server_negotiation.cc:407] Connection header received
0925 12:25:59.828024 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.828037 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.828059 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.828091 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.829465 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.830229 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.831788 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.832131 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.832192 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.834572 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.834573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.834574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.834582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.836425 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.837033 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.837034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.837034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.837108 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.837424 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.837425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.837426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.837577 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.837679 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.838001 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.838096 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:25:59.839903 23705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:25:59.840975 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.827601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.1.47.189:37911)
0925 12:25:59.827681 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.827683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.829535 (+ 1852us) server_negotiation.cc:407] Connection header received
0925 12:25:59.829691 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.829692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.829715 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.829745 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.830744 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.831480 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.832915 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.833708 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.833757 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.836727 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.836729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.836729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.836737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.838573 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.839502 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.839504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.839504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.839592 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.840055 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.840056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.840057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.840275 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.840405 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.840720 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.840845 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 12:25:59.841763 23637 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [term 1 FOLLOWER]: Refusing update from remote peer 4a44fa1647254ebcba133e2220dd1eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:59.842259 23703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:59.843688 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a44fa1647254ebcba133e2220dd1eaa. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.843781 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.853544 23684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.853667 23684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: This master's current role is: LEADER
I20260925 12:25:59.854265 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.854351 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.854365 23707 mvcc.cc:205] Tried to move back new op lower bound from 7333229198708838400 to 7333229198643515392. Current Snapshot: MvccSnapshot[applied={T|T < 7333229198708838400}]
I20260925 12:25:59.854601 23708 mvcc.cc:205] Tried to move back new op lower bound from 7333229198708838400 to 7333229198643515392. Current Snapshot: MvccSnapshot[applied={T|T < 7333229198708838400}]
I20260925 12:25:59.856367 23705 catalog_manager.cc:1347] Generated new cluster ID: 86a291431e5c40ed8bf697ab079a1c4c
I20260925 12:25:59.856422 23705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:25:59.857045 23588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } attempt: 3
Sep 25 12:25:59 dist-test-slave-l9ww krb5kdc[23432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:25:59.864781 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.857295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:33145 (local address 127.0.0.1:59290)
0925 12:25:59.857339 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 12:25:59.857350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:25:59.857385 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:25:59.857528 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:25:59.857529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:25:59.857712 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:25:59.857758 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.857759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.858685 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.858686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.859469 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:25:59.859471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.859962 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.859963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:25:59.860032 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.860557 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:25:59.860562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:25:59.862005 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:25:59.863808 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.863809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.863810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.864027 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.864210 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:25:59.864210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:25:59.864211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:25:59.864258 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:25:59.864488 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:25:59.864488 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:25:59.864530 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 12:25:59.864596 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 12:25:59.864687 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 12:25:59.865105 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:25:59.857343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.1.47.188:33145)
0925 12:25:59.857392 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.857394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.857405 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:25:59.857441 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.857441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.857452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.857525 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.857813 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.858600 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.859548 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:25:59.859894 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:25:59.859967 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:25:59.862092 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:25:59.862094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:25:59.862094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:25:59.862116 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:25:59.863728 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.864098 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.864099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.864100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.864151 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:25:59.864310 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:25:59.864310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:25:59.864311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:25:59.864422 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:25:59.864469 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:25:59.864945 (+ 476us) server_negotiation.cc:299] Negotiation successful
0925 12:25:59.865022 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:25:59.866495 23588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: Bootstrap starting.
I20260925 12:25:59.867255 23588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: Neither blocks nor log segments found. Creating new log.
I20260925 12:25:59.867548 23588 log.cc:824] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: Log is configured to *not* fsync() on all Append() calls
I20260925 12:25:59.868611 23588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: No bootstrap required, opened a new log
I20260925 12:25:59.869904 23705 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:25:59.870618 23705 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:25:59.872808 23588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.873152 23588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0263740cad6d42ed95992087cb050f50, State: Initialized, Role: FOLLOWER
I20260925 12:25:59.873369 23588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } }
I20260925 12:25:59.874071 23562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a44fa1647254ebcba133e2220dd1eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0263740cad6d42ed95992087cb050f50"
I20260925 12:25:59.874184 23562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:25:59.875083 23562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a44fa1647254ebcba133e2220dd1eaa in term 1.
I20260925 12:25:59.875541 23561 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [term 1 FOLLOWER]: Refusing update from remote peer 4a44fa1647254ebcba133e2220dd1eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:25:59.875800 23684 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:25:59.876672 23705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa: Generated new TSK 0
I20260925 12:25:59.876847 23705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:25:59.877452 23588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:25:59.877960 23703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.878057 23703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a44fa1647254ebcba133e2220dd1eaa [sys.catalog]: This master's current role is: LEADER
W20260925 12:25:59.880443 23728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:25:59.884450 23728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:25:59.883957 23715 mvcc.cc:205] Tried to move back new op lower bound from 7333229198849081344 to 7333229198643515392. Current Snapshot: MvccSnapshot[applied={T|T < 7333229198708838400}]
I20260925 12:25:59.884686 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.884837 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.884919 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a44fa1647254ebcba133e2220dd1eaa. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.885015 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:25:59.885082 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a44fa1647254ebcba133e2220dd1eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a44fa1647254ebcba133e2220dd1eaa" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 41599 } } peers { permanent_uuid: "0263740cad6d42ed95992087cb050f50" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 37911 } } peers { permanent_uuid: "268d928d936e47f5a79aca807383b5e7" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 33145 } } }
I20260925 12:25:59.885213 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:25:59.902757 23668 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.901374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.1.47.188:33145)
0925 12:25:59.901429 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.901432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.902036 (+ 604us) server_negotiation.cc:407] Connection header received
0925 12:25:59.902160 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.902162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.902173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.902203 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.902679 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59304: BlockingRecv error: recv got EOF from 127.0.0.1:59304 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 12:25:59.903505 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.901777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.1.47.189:37911)
0925 12:25:59.901817 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.901820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.902992 (+ 1172us) server_negotiation.cc:407] Connection header received
0925 12:25:59.903100 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.903101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.903111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.903142 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.903445 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39340: BlockingRecv error: recv got EOF from 127.0.0.1:39340 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 12:25:59.904451 23516 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.903884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.1.47.190:41599)
0925 12:25:59.903933 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.903935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.903948 (+ 13us) server_negotiation.cc:407] Connection header received
0925 12:25:59.904014 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.904014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.904024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.904103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.904380 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55126: BlockingRecv error: recv got EOF from 127.0.0.1:55126 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 12:25:59.906491 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.905754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.1.47.189:37911)
0925 12:25:59.905789 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.905791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.905972 (+ 181us) server_negotiation.cc:407] Connection header received
0925 12:25:59.906078 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.906078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.906110 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.906140 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.906445 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39352: BlockingRecv error: recv got EOF from 127.0.0.1:39352 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 12:25:59.909482 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.908738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.1.47.189:37911)
0925 12:25:59.908771 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.908772 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.908986 (+ 214us) server_negotiation.cc:407] Connection header received
0925 12:25:59.909078 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.909079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.909101 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.909130 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.909439 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39354: BlockingRecv error: recv got EOF from 127.0.0.1:39354 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 12:25:59.913313 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.912606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.1.47.189:37911)
0925 12:25:59.912640 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.912642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.912845 (+ 203us) server_negotiation.cc:407] Connection header received
0925 12:25:59.912933 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.912934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.912943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.912970 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.913268 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39366: BlockingRecv error: recv got EOF from 127.0.0.1:39366 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 12:25:59.919389 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.918533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.1.47.189:37911)
0925 12:25:59.918572 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.918574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.918824 (+ 250us) server_negotiation.cc:407] Connection header received
0925 12:25:59.918920 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.918922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.918934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.918963 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.919339 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39382: BlockingRecv error: recv got EOF from 127.0.0.1:39382 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 12:25:59.929739 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.928799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.1.47.189:37911)
0925 12:25:59.928843 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.928845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.929134 (+ 289us) server_negotiation.cc:407] Connection header received
0925 12:25:59.929243 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.929244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.929257 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.929288 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.929679 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39394: BlockingRecv error: recv got EOF from 127.0.0.1:39394 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 12:25:59.948788 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.947608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.1.47.189:37911)
0925 12:25:59.947661 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.947663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.948016 (+ 353us) server_negotiation.cc:407] Connection header received
0925 12:25:59.948161 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.948162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.948178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.948219 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.948717 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39396: BlockingRecv error: recv got EOF from 127.0.0.1:39396 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260925 12:25:59.983593 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:25:59.982421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.1.47.189:37911)
0925 12:25:59.982478 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 12:25:59.982480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:25:59.982837 (+ 357us) server_negotiation.cc:407] Connection header received
0925 12:25:59.982977 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:25:59.982979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:25:59.982994 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:25:59.983034 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:25:59.983507 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39404: BlockingRecv error: recv got EOF from 127.0.0.1:39404 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260925 12:26:00.049912 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:00.049268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.1.47.189:37911)
0925 12:26:00.049340 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.049343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.049403 (+ 60us) server_negotiation.cc:407] Connection header received
0925 12:26:00.049435 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.049436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.049449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.049508 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:00.049848 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39420: BlockingRecv error: recv got EOF from 127.0.0.1:39420 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 12:26:00.179854 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:00.179186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39432 (local address 127.1.47.189:37911)
0925 12:26:00.179248 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.179250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.179347 (+ 97us) server_negotiation.cc:407] Connection header received
0925 12:26:00.179395 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.179396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.179408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.179476 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:00.179782 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39432: BlockingRecv error: recv got EOF from 127.0.0.1:39432 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260925 12:26:00.438082 23592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:00.437310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.1.47.189:37911)
0925 12:26:00.437394 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.437397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.437435 (+ 38us) server_negotiation.cc:407] Connection header received
0925 12:26:00.437542 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.437544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.437556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.437644 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:00.438015 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39438: BlockingRecv error: recv got EOF from 127.0.0.1:39438 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 12:26:00.812829 23681 catalog_manager.cc:1259] Loaded cluster ID: 86a291431e5c40ed8bf697ab079a1c4c
I20260925 12:26:00.812927 23681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: loading cluster ID for follower catalog manager: success
I20260925 12:26:00.813737 23681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: acquiring CA information for follower catalog manager: success
I20260925 12:26:00.814056 23681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 268d928d936e47f5a79aca807383b5e7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:26:00.884963 23728 catalog_manager.cc:1259] Loaded cluster ID: 86a291431e5c40ed8bf697ab079a1c4c
I20260925 12:26:00.885111 23728 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: loading cluster ID for follower catalog manager: success
I20260925 12:26:00.885938 23728 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: acquiring CA information for follower catalog manager: success
I20260925 12:26:00.886204 23728 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0263740cad6d42ed95992087cb050f50: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:26:00.954435 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:00.951450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.1.47.189:37911)
0925 12:26:00.951588 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.951591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.951640 (+ 49us) server_negotiation.cc:407] Connection header received
0925 12:26:00.951754 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.951755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.951827 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.951956 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:00.952145 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.952662 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.953496 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.953702 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.953800 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:00.954003 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:00.954126 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:00.954219 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:00.954279 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 12:26:00.954302 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260925 12:26:00.958992 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:00.956636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.1.47.188:33145)
0925 12:26:00.956777 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.956779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.956787 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:26:00.956831 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.956832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.956885 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.956969 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:00.957052 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.957549 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.958345 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.958538 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.958593 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:00.958714 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:00.958821 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:00.958879 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:00.958911 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 12:26:00.958927 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260925 12:26:00.963577 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:00.960813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.1.47.188:33145)
0925 12:26:00.960892 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.960895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.961069 (+ 174us) server_negotiation.cc:407] Connection header received
0925 12:26:00.961117 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.961118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.961129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.961176 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:00.961351 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.962052 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.962800 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.963074 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.963137 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:00.963293 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:00.963330 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:00.963361 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:00.963477 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 12:26:00.963504 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 12:26:00.963763 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:00.960806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.1.47.189:37911)
0925 12:26:00.960858 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.960861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.960914 (+ 53us) server_negotiation.cc:407] Connection header received
0925 12:26:00.961020 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.961021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.961062 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.961112 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:00.961265 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.962015 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.963004 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.963311 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.963378 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:00.963507 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:00.963542 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:00.963593 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:00.963668 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 12:26:00.963689 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 12:26:00.964185 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:00.960874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.1.47.190:41599)
0925 12:26:00.960974 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:00.960976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:00.960984 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:26:00.961029 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:00.961030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:00.961085 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:00.961187 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:00.961313 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.961818 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.962853 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:00.963167 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:00.963248 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:00.963565 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:00.963800 (+ 235us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:00.963963 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:00.964048 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 12:26:00.964073 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
I20260925 12:26:00.965999 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23445
I20260925 12:26:00.971493 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23520
I20260925 12:26:00.977058 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23595
2026-09-25T12:26:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2828 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 12:26:01 dist-test-slave-l9ww krb5kdc[23813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 12:26:01 dist-test-slave-l9ww krb5kdc[23813](info): set up 2 sockets
Sep 25 12:26:01 dist-test-slave-l9ww krb5kdc[23813](info): commencing operation
krb5kdc: starting...
W20260925 12:26:02.092247 1214 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/test-user.keytab.
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T12:26:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T12:26:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
Entry for principal kudu/127.1.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab.
I20260925 12:26:02.212961 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42579
--webserver_interface=127.1.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 12:26:02.300542 23826 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:26:02.300768 23826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:26:02.300787 23826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:26:02.300808 23826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:26:02.302009 23826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:26:02.302054 23826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:26:02.302068 23826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:26:02.302081 23826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:26:02.302093 23826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:26:02.303606 23826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.190.keytab
--principal=kudu/127.1.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.190:42579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:26:02.303870 23826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:26:02.304157 23826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:26:02.309230 23831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:26:02.309237 23832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:26:02.309427 23826 server_base.cc:1034] running on GCE node
W20260925 12:26:02.309293 23834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:26:02.309748 23826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:26:02.310045 23826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:26:02.311231 23826 hybrid_clock.cc:648] HybridClock initialized: now 1790339162311216 us; error 28 us; skew 500 ppm
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:26:02.314157 23826 init.cc:377] Logged in from keytab as kudu/127.1.47.190@KRBTEST.COM (short username kudu)
I20260925 12:26:02.314692 23826 webserver.cc:460] Webserver started at http://127.1.47.190:39543/ using document root <none> and password file <none>
I20260925 12:26:02.314952 23826 fs_manager.cc:359] Metadata directory not provided
I20260925 12:26:02.315027 23826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:26:02.315198 23826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:26:02.316459 23826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data/instance:
uuid: "79ffa656a4204ce3bfa4e1f49870311e"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "75257d64138cd2b71414d367f694f9dd"
server_key_iv: "83c6910be51493a9e675ac40f01d9331"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.316866 23826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance:
uuid: "79ffa656a4204ce3bfa4e1f49870311e"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "75257d64138cd2b71414d367f694f9dd"
server_key_iv: "83c6910be51493a9e675ac40f01d9331"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.318311 23826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 12:26:02.319175 23841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:26:02.319404 23826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 12:26:02.319525 23826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal
uuid: "79ffa656a4204ce3bfa4e1f49870311e"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "75257d64138cd2b71414d367f694f9dd"
server_key_iv: "83c6910be51493a9e675ac40f01d9331"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.319612 23826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:26:02.337108 23826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:26:02.339778 23826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:26:02.340034 23826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:26:02.344409 23826 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.190:42579
I20260925 12:26:02.344447 23893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.190:42579 every 8 connection(s)
I20260925 12:26:02.344864 23826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/data/info.pb
I20260925 12:26:02.345907 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:26:02.347487 23894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:26:02.348691 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23826
I20260925 12:26:02.348788 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-0/wal/instance
I20260925 12:26:02.348989 1214 external_mini_cluster.cc:1428] Setting key 5f0f574e39a6f89d3e3ef94ddcbed3f7
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.189@KRBTEST.COM; defaulting to no policy
I20260925 12:26:02.359385 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.348328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.1.47.190:42579)
0925 12:26:02.348599 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.348603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.348632 (+ 29us) server_negotiation.cc:407] Connection header received
0925 12:26:02.349436 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.349438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.349457 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.349552 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.350375 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.351753 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.352719 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.353248 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.353370 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.355872 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.355881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.355881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.355890 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.357800 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.358173 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.358176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.358176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.358267 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.358572 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.358573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.358573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.358814 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.358884 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.359078 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.359227 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
I20260925 12:26:02.359396 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.348302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42579 (local address 127.0.0.1:56446)
0925 12:26:02.348491 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.348555 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.349248 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.349584 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.349587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.349997 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.350231 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.350237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.351847 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.351848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.352587 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.352589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.353357 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.353358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.353415 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.354018 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.354027 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.355752 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.357875 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.357882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.357884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.358079 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.358336 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.358336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.358337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.358463 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.358881 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.358887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.358953 (+ 66us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.359031 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.359166 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.189@KRBTEST.COM" created.
I20260925 12:26:02.360311 23894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:26:02.360700 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.360562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47556)
0925 12:26:02.360616 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.360667 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 12:26:02.360845 23845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
I20260925 12:26:02.361320 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.361214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47566)
0925 12:26:02.361261 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.361290 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:26:02.361433 23894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:41355: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
Entry for principal kudu/127.1.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab.
I20260925 12:26:02.375586 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:41355
--webserver_interface=127.1.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:26:02.410686 23894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } attempt: 1
I20260925 12:26:02.411278 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.411125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47576)
0925 12:26:02.411194 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.411242 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 12:26:02.411662 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.411568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47592)
0925 12:26:02.411614 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.411642 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 12:26:02.411763 23894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:41355: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
W20260925 12:26:02.462462 23901 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:26:02.462702 23901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:26:02.462720 23901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:26:02.462742 23901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:26:02.464031 23901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:26:02.464076 23901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:26:02.464092 23901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:26:02.464103 23901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:26:02.464116 23901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:26:02.465668 23901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.189.keytab
--principal=kudu/127.1.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.189:41355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:26:02.465945 23901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:26:02.466190 23901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:26:02.471055 23906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:26:02.471077 23909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:26:02.471062 23907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:26:02.471312 23901 server_base.cc:1034] running on GCE node
I20260925 12:26:02.471560 23901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:26:02.471907 23901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:26:02.473074 23901 hybrid_clock.cc:648] HybridClock initialized: now 1790339162473048 us; error 46 us; skew 500 ppm
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:26:02.475811 23901 init.cc:377] Logged in from keytab as kudu/127.1.47.189@KRBTEST.COM (short username kudu)
I20260925 12:26:02.476292 23901 webserver.cc:460] Webserver started at http://127.1.47.189:34959/ using document root <none> and password file <none>
I20260925 12:26:02.476523 23901 fs_manager.cc:359] Metadata directory not provided
I20260925 12:26:02.476593 23901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:26:02.476751 23901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:26:02.477860 23901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data/instance:
uuid: "34d9dd3373b44eb8b98c9c41433c5160"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "8fc13526f973b136f683f5cc0daf255b"
server_key_iv: "527d523a771bcf3d8b68668985b165cf"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.478245 23901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal/instance:
uuid: "34d9dd3373b44eb8b98c9c41433c5160"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "8fc13526f973b136f683f5cc0daf255b"
server_key_iv: "527d523a771bcf3d8b68668985b165cf"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.479696 23901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 12:26:02.479912 23894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } attempt: 2
I20260925 12:26:02.480321 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.480183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47594)
0925 12:26:02.480241 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.480284 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:26:02.480715 23916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:26:02.480783 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.480646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47602)
0925 12:26:02.480702 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.480739 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 12:26:02.480876 23894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.189:41355: Network error: Client connection negotiation failed: client connection to 127.1.47.189:41355: connect: Connection refused (error 111)
I20260925 12:26:02.480930 23901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 12:26:02.481015 23901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal
uuid: "34d9dd3373b44eb8b98c9c41433c5160"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "8fc13526f973b136f683f5cc0daf255b"
server_key_iv: "527d523a771bcf3d8b68668985b165cf"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.481070 23901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:26:02.494230 23901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:26:02.496141 23901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:26:02.496253 23901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:26:02.500401 23901 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.189:41355
I20260925 12:26:02.500519 23968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.189:41355 every 8 connection(s)
I20260925 12:26:02.500959 23901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/data/info.pb
I20260925 12:26:02.502014 23969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:26:02.503582 23969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:26:02.510821 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23901
I20260925 12:26:02.510910 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-1/wal/instance
I20260925 12:26:02.511137 1214 external_mini_cluster.cc:1428] Setting key a5eb1f0cd3599b1cdca9dfe627850f71
I20260925 12:26:02.513342 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.504339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.1.47.190:42579)
0925 12:26:02.504397 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.504399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.504576 (+ 177us) server_negotiation.cc:407] Connection header received
0925 12:26:02.505080 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.505082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.505094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.505165 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.505922 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.506498 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.507516 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.507752 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.507817 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.510044 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.510045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.510045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.510052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.511702 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.512237 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.512238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.512238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.512294 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.512609 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.512611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.512611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.512771 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.512849 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.513137 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.513235 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:26:02.513415 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.504280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42579 (local address 127.0.0.1:56458)
0925 12:26:02.504441 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.504484 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.504934 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.505253 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.505256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.505610 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.505812 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.505816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.506599 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.506599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.507412 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.507414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.507845 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.507846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.508008 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.508561 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.508577 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.509934 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.511787 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.511795 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.511797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.512158 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.512366 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.512367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.512368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.512526 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.512843 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.512848 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.512971 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.513086 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.513225 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260925 12:26:02.514051 23969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.47.188@KRBTEST.COM" created.
I20260925 12:26:02.521986 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.514361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47614)
0925 12:26:02.514417 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.514429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.514475 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.514712 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.514713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.514895 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.514963 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.514965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.515802 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.515803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.516756 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.516759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.517215 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.517216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.517297 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.517677 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.517682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.518971 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.520769 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.520770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.520772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.520941 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.521249 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.521249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.521250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.521307 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.521653 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.521654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.521693 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.521767 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.521869 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 12:26:02.522217 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.514399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.1.47.189:41355)
0925 12:26:02.514577 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.514581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.514595 (+ 14us) server_negotiation.cc:407] Connection header received
0925 12:26:02.514613 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.514615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.514626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.514687 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.515049 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.515708 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.516841 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.517133 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.517188 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.519069 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.519079 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.519080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.519091 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.520675 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.521057 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.521060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.521060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.521107 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.521383 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.521384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.521384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.521557 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.521674 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.521806 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.522123 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":239,"server-negotiator.queue_time_us":143,"thread_start_us":117,"threads_started":1}
I20260925 12:26:02.522904 23969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:26:02.523279 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.523169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47010)
0925 12:26:02.523220 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.523261 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:26:02.523437 23918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
I20260925 12:26:02.523914 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.523786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47014)
0925 12:26:02.523837 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.523888 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 12:26:02.524050 23969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:35597: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
Entry for principal kudu/127.1.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab.
I20260925 12:26:02.538767 1214 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuGajnc/build/release/bin/kudu
/tmp/dist-test-taskuGajnc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:35597
--webserver_interface=127.1.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.47.148:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 12:26:02.572046 23894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } attempt: 3
I20260925 12:26:02.573287 23969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } attempt: 1
I20260925 12:26:02.573664 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.573532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47022)
0925 12:26:02.573588 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.573627 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 12:26:02.573989 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.573887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47036)
0925 12:26:02.573940 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.573968 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 12:26:02.574081 23969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:35597: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:26:02.579691 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.572492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47624)
0925 12:26:02.572543 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.572564 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.572639 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.572797 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.572798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.572965 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.573028 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.573030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.573862 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.573863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.574538 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.574541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.574956 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.574957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.575036 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.575572 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.575576 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.577028 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.578618 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.578619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.578620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.578794 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.579018 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.579018 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.579019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.579080 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.579358 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.579359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.579405 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.579490 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.579592 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 12:26:02.579691 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.572559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.1.47.189:41355)
0925 12:26:02.572627 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.572630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.572641 (+ 11us) server_negotiation.cc:407] Connection header received
0925 12:26:02.572708 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.572709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.572720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.572791 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.573122 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.573780 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.574640 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.574880 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.574959 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.577162 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.577164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.577164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.577172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.578514 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.578884 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.578885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.578885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.578944 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.579159 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.579159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.579160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.579278 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.579352 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.579514 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.579592 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 12:26:02.580107 23894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:26:02.580458 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.580354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47052)
0925 12:26:02.580404 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.580436 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 12:26:02.580821 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.580740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47056)
0925 12:26:02.580782 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.580805 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 12:26:02.580920 23894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:35597: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
I20260925 12:26:02.612072 23894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } attempt: 1
I20260925 12:26:02.612623 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.612469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47068)
0925 12:26:02.612541 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.612594 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 12:26:02.613042 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.612969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47072)
0925 12:26:02.612998 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.613023 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 12:26:02.613199 23894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:35597: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
W20260925 12:26:02.622995 23976 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 12:26:02.623229 23976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 12:26:02.623246 23976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 12:26:02.623267 23976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 12:26:02.624553 23976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 12:26:02.624598 23976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 12:26:02.624611 23976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 12:26:02.624624 23976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 12:26:02.624636 23976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 12:26:02.626107 23976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.47.148:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.47.190:42579,127.1.47.189:41355,127.1.47.188:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/krb5kdc/kudu_127.1.47.188.keytab
--principal=kudu/127.1.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.47.188:35597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 11:20:57 UTC on ccycloud.quasar-ptjzfi.root.comops.site
build id 4672270
I20260925 12:26:02.626376 23976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 12:26:02.626618 23976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 12:26:02.631460 23984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:26:02.631449 23982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 12:26:02.631502 23981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 12:26:02.631736 23976 server_base.cc:1034] running on GCE node
I20260925 12:26:02.632136 23976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 12:26:02.632417 23976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 12:26:02.633600 23976 hybrid_clock.cc:648] HybridClock initialized: now 1790339162633571 us; error 40 us; skew 500 ppm
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 12:26:02.636132 23976 init.cc:377] Logged in from keytab as kudu/127.1.47.188@KRBTEST.COM (short username kudu)
I20260925 12:26:02.636579 23976 webserver.cc:460] Webserver started at http://127.1.47.188:43309/ using document root <none> and password file <none>
I20260925 12:26:02.636812 23976 fs_manager.cc:359] Metadata directory not provided
I20260925 12:26:02.636859 23976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 12:26:02.637028 23976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 12:26:02.638123 23976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data/instance:
uuid: "7b120bc82c7b4a1fbc20c96932fc98de"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "b0fbb7616083b78ed886772e08455f5c"
server_key_iv: "ec27b158e5def689b616e693698a70b7"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.638517 23976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal/instance:
uuid: "7b120bc82c7b4a1fbc20c96932fc98de"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "b0fbb7616083b78ed886772e08455f5c"
server_key_iv: "ec27b158e5def689b616e693698a70b7"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.640038 23976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 12:26:02.640797 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 12:26:02.640969 23976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 12:26:02.641074 23976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data,/tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal
uuid: "7b120bc82c7b4a1fbc20c96932fc98de"
format_stamp: "Formatted at 2026-09-25 12:26:02 on dist-test-slave-l9ww"
server_key: "b0fbb7616083b78ed886772e08455f5c"
server_key_iv: "ec27b158e5def689b616e693698a70b7"
server_key_version: "encryptionkey@0"
I20260925 12:26:02.641152 23976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 12:26:02.642244 23969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } attempt: 2
I20260925 12:26:02.642678 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.642527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47088)
0925 12:26:02.642596 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.642639 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 12:26:02.643160 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.643035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47090)
0925 12:26:02.643088 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.643127 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 12:26:02.643368 23969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.47.188:35597: Network error: Client connection negotiation failed: client connection to 127.1.47.188:35597: connect: Connection refused (error 111)
I20260925 12:26:02.655812 23976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 12:26:02.658056 23976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 12:26:02.658215 23976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 12:26:02.662438 23976 rpc_server.cc:307] RPC server started. Bound to: 127.1.47.188:35597
I20260925 12:26:02.662506 24043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.47.188:35597 every 8 connection(s)
I20260925 12:26:02.662853 23976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/data/info.pb
I20260925 12:26:02.663738 1214 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuGajnc/build/release/bin/kudu as pid 23976
I20260925 12:26:02.663820 1214 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuGajnc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790338888974512-1214-0/minicluster-data/master-2/wal/instance
I20260925 12:26:02.663946 24044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 12:26:02.664119 1214 external_mini_cluster.cc:1428] Setting key 9ad19d4b4aa99da4f2ac5d04226f7576
I20260925 12:26:02.666317 24044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:26:02.675706 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.667036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.1.47.190:42579)
0925 12:26:02.667094 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.667096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.667233 (+ 137us) server_negotiation.cc:407] Connection header received
0925 12:26:02.667315 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.667316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.667324 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.667373 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.668019 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.668758 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.669906 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.670122 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.670223 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.672730 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.672732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.672732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.672740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.674425 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.674806 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.674807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.674807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.674872 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.675106 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.675107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.675107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.675244 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.675323 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.675521 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.675604 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 12:26:02.675902 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.664648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.1.47.188:35597)
0925 12:26:02.664841 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.664849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.664874 (+ 25us) server_negotiation.cc:407] Connection header received
0925 12:26:02.665409 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.665416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.665464 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.665588 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.666417 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.667927 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.668975 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.669589 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.669714 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.672287 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.672295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.672295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.672301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.674261 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.674680 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.674684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.674684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.674786 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.674996 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.674997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.674997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.675249 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.675394 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.675493 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.675642 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260925 12:26:02.676014 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.666932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.190:42579 (local address 127.0.0.1:56470)
0925 12:26:02.667115 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.667158 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.667249 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.667414 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.667417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.667763 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.667921 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.667927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.668876 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.668877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.669815 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.669817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.670273 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.670275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.670370 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.670834 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.670843 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.672626 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.674511 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.674516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.674519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.674717 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.674948 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.674949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.674949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.675037 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.675315 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.675321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.675399 (+ 78us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.675494 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.675840 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":69,"thread_start_us":97,"threads_started":1}
I20260925 12:26:02.676653 24044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.190@KRBTEST.COM
I20260925 12:26:02.683559 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.677023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.1.47.189:41355)
0925 12:26:02.677065 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.677066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.677073 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:26:02.677095 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.677096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.677105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.677158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.677486 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.677951 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.678663 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.678882 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.678958 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.680858 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.680860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.680860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.680867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.682463 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.682779 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.682781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.682781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.682842 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.683025 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.683026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.683026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.683173 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.683255 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.683416 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.683493 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 12:26:02.683611 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.676919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.189:41355 (local address 127.0.0.1:47636)
0925 12:26:02.676955 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.676966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.676998 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.677186 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.677187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.677343 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.677395 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.677397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.678060 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.678061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.678574 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.678575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.678953 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.678954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.679014 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.679461 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.679466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.680757 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.682545 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.682546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.682546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.682705 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.682917 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.682918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.682918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.682966 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.683259 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.683260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.683307 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.683387 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.683489 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 12:26:02.684091 24044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260925 12:26:02.684695 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.676819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.1.47.190:42579)
0925 12:26:02.676861 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.676864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.676889 (+ 25us) server_negotiation.cc:407] Connection header received
0925 12:26:02.676909 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.676910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.676920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.676968 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.677581 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.678043 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.678730 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.678934 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.679020 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.681510 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.681512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.681512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.681520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.683034 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.683461 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.683462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.683463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.683519 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.683738 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.683739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.683739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.683901 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.684095 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.684518 (+ 423us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.684600 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.188@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:26:02.688357 23894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } attempt: 2
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.47.189@KRBTEST.COM
I20260925 12:26:02.691716 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.684402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47118)
0925 12:26:02.684444 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.684454 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.684490 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.684622 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.684623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.684732 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.684760 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.684761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.685675 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.685676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.686312 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.686314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.686754 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.686755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.686809 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.687142 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.687147 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.688479 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.690452 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.690453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.690453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.690634 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.690854 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.690855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.690856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.690917 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.691418 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.691419 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.691459 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.691514 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.691599 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 12:26:02.692019 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.684456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.1.47.188:35597)
0925 12:26:02.684497 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.684498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.684508 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:26:02.684542 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.684543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.684552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.684610 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.684889 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.685603 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.686384 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.686686 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.686735 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.688574 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.688575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.688575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.688582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.690369 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.690729 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.690730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.690731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.690791 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.690983 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.690983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.690984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.691333 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.691397 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.691835 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.691938 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:26:02.693223 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.685363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.1.47.189:41355)
0925 12:26:02.685399 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.685401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.685410 (+ 9us) server_negotiation.cc:407] Connection header received
0925 12:26:02.685432 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.685433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.685441 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.685494 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.686004 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.686488 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.687340 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.687516 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.687567 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.690358 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.690359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.690359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.690367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.691798 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.692277 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.692278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.692279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.692345 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.692584 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.692585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.692585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.692713 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.692890 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.693028 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.693108 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 12:26:02.694221 24044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: Bootstrap starting.
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.190@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:26:02.694702 1214 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 12:26:02.695029 24044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: Neither blocks nor log segments found. Creating new log.
I20260925 12:26:02.695643 24044 log.cc:824] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: Log is configured to *not* fsync() on all Append() calls
I20260925 12:26:02.696898 24044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: No bootstrap required, opened a new log
I20260925 12:26:02.696123 23868 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "79ffa656a4204ce3bfa4e1f49870311e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56476
I20260925 12:26:02.698307 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.688627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47132)
0925 12:26:02.688674 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.688684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.688722 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.689305 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.689306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.689463 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.689507 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.689509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.690891 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.690892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.691720 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.691722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.692662 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.692663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.692844 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.693325 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.693332 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.694902 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.697025 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.697026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.697027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.697277 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.697503 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.697504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.697504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.697582 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.697939 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.697940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.697990 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.698087 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.698197 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 12:26:02.698351 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.688687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.1.47.188:35597)
0925 12:26:02.689152 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.689154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.689162 (+ 8us) server_negotiation.cc:407] Connection header received
0925 12:26:02.689186 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.689187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.689198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.689278 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.689607 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.690773 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.691954 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.692605 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.692662 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.694990 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.694991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.694992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.694999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.696859 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.697365 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.697366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.697367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.697433 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.697661 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.697662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.697662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.697860 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.697934 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.698131 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.698212 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":375,"threads_started":1}
I20260925 12:26:02.698835 24044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.698995 24044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b120bc82c7b4a1fbc20c96932fc98de, State: Initialized, Role: FOLLOWER
I20260925 12:26:02.699070 24044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.699343 24051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.699409 24044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:26:02.699431 24051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.699813 23894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: Bootstrap starting.
I20260925 12:26:02.700621 23894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: Neither blocks nor log segments found. Creating new log.
I20260925 12:26:02.700992 23894 log.cc:824] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: Log is configured to *not* fsync() on all Append() calls
I20260925 12:26:02.701838 23894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: No bootstrap required, opened a new log
I20260925 12:26:02.703301 23894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.703445 23894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ffa656a4204ce3bfa4e1f49870311e, State: Initialized, Role: FOLLOWER
I20260925 12:26:02.703536 23894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.703917 23894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:26:02.701889 23976 master_runner.cc:186] Error getting master registration for 127.1.47.190:42579: OK, instance_id { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" instance_seqno: 1790339162343183 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:26:02.704149 23976 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 12:26:02.704226 24062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:26:02.704262 24062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:26:02.704392 23868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 12:26:02.704478 23868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:26:02.705411 23868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.705693 23868 leader_election.cc:288] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [CANDIDATE]: Term 1 election: Requested vote from peers 34d9dd3373b44eb8b98c9c41433c5160 (127.1.47.189:41355), 7b120bc82c7b4a1fbc20c96932fc98de (127.1.47.188:35597)
I20260925 12:26:02.706173 24018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79ffa656a4204ce3bfa4e1f49870311e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7b120bc82c7b4a1fbc20c96932fc98de"
I20260925 12:26:02.706290 24018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [term 0 FOLLOWER]: Advancing to term 1
W20260925 12:26:02.706727 24076 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:26:02.706774 24076 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:26:02.707316 24018 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79ffa656a4204ce3bfa4e1f49870311e in term 1.
I20260925 12:26:02.707578 23843 leader_election.cc:302] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79ffa656a4204ce3bfa4e1f49870311e, 7b120bc82c7b4a1fbc20c96932fc98de; no voters:
I20260925 12:26:02.708297 24065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.708387 24065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.708438 24065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 12:26:02.708478 24065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [term 1 LEADER]: Becoming Leader. State: Replica: 79ffa656a4204ce3bfa4e1f49870311e, State: Running, Role: LEADER
I20260925 12:26:02.708557 24065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.708985 24065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ffa656a4204ce3bfa4e1f49870311e. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.709082 24065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: This master's current role is: LEADER
I20260925 12:26:02.709357 24087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 12:26:02.709707 24087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 12:26:02.708703 23826 master_runner.cc:186] Error getting master registration for 127.1.47.189:41355: OK, instance_id { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" instance_seqno: 1790339162498951 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 12:26:02.711647 23826 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 12:26:02.722712 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.707954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.1.47.189:41355)
0925 12:26:02.708003 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.708006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.709917 (+ 1911us) server_negotiation.cc:407] Connection header received
0925 12:26:02.710093 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.710094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.710107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.710155 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.711017 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.711770 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.713258 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.713566 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.713631 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.717584 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.717585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.717585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.717593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.719551 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.720360 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.720361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.720362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.720426 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.720645 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.720646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.720646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.720788 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.721379 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.721674 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.721759 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 12:26:02.725034 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.707097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.1.47.190:42579)
0925 12:26:02.707141 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.707143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.711127 (+ 3984us) server_negotiation.cc:407] Connection header received
0925 12:26:02.713308 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.713310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.713323 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.713358 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.714156 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.714873 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.716881 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.717180 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.717255 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.720673 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.720675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.720675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.720682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.722613 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.723988 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.723989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.724000 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.724059 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.724349 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.724350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.724350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.724493 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.724571 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.724861 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.724939 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 12:26:02.725855 24018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [term 1 FOLLOWER]: Refusing update from remote peer 79ffa656a4204ce3bfa4e1f49870311e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:26:02.726239 24084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:26:02.727983 24051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ffa656a4204ce3bfa4e1f49870311e. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.728109 24051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.728490 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.708042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.1.47.188:35597)
0925 12:26:02.708093 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.708095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.711217 (+ 3122us) server_negotiation.cc:407] Connection header received
0925 12:26:02.711319 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.711321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.711332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.711367 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.712279 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.713019 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.717857 (+ 4838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.718245 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.718293 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.721868 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.721869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.721870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.721878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.723818 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.724401 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.724403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.724403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.724468 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.724700 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.724701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.724701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.724834 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.724932 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.728288 (+ 3356us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.728394 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 12:26:02.729945 24065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.730042 24065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: This master's current role is: LEADER
I20260925 12:26:02.730206 24051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.730296 24051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.730495 24089 mvcc.cc:205] Tried to move back new op lower bound from 7333229210521042944 to 7333229210454990848. Current Snapshot: MvccSnapshot[applied={T|T < 7333229210521042944}]
I20260925 12:26:02.731392 24087 catalog_manager.cc:1347] Generated new cluster ID: d6505c08ef22432985f3d95d562ebdf8
I20260925 12:26:02.731441 24087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 12:26:02.734490 23969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } attempt: 3
Sep 25 12:26:02 dist-test-slave-l9ww krb5kdc[23813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790339162, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.47.189@KRBTEST.COM for kudu/127.1.47.188@KRBTEST.COM
I20260925 12:26:02.745801 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.734842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.1.47.188:35597)
0925 12:26:02.734883 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.734885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.734895 (+ 10us) server_negotiation.cc:407] Connection header received
0925 12:26:02.734980 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.734981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.734991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.735039 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.735436 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.736309 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.737576 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.737929 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.737994 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.741827 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 12:26:02.741828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 12:26:02.741828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 12:26:02.741854 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 12:26:02.743973 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.744489 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.744490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.744490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.744565 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 12:26:02.744839 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 12:26:02.744840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 12:26:02.744840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 12:26:02.745005 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 12:26:02.745083 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:02.745612 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 12:26:02.745700 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 12:26:02.746176 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:02.734759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.47.188:35597 (local address 127.0.0.1:47144)
0925 12:26:02.734844 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 12:26:02.734856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 12:26:02.734903 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 12:26:02.735099 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 12:26:02.735100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 12:26:02.735300 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 12:26:02.735350 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.735352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.736517 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.736518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.737453 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 12:26:02.737456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 12:26:02.738083 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 12:26:02.738084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 12:26:02.738166 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:02.738841 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 12:26:02.738847 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 12:26:02.741707 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 12:26:02.744100 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.744101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.744101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.744355 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.744659 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 12:26:02.744660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 12:26:02.744660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 12:26:02.744747 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 12:26:02.745133 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 12:26:02.745134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 12:26:02.745206 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 12:26:02.745279 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 12:26:02.745393 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 12:26:02.748468 23969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: Bootstrap starting.
I20260925 12:26:02.748983 24087 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 12:26:02.749326 23969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: Neither blocks nor log segments found. Creating new log.
I20260925 12:26:02.749646 23969 log.cc:824] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: Log is configured to *not* fsync() on all Append() calls
I20260925 12:26:02.749719 24087 catalog_manager.cc:1504] Loading token signing keys...
I20260925 12:26:02.752055 23969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: No bootstrap required, opened a new log
I20260925 12:26:02.753579 24087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e: Generated new TSK 0
I20260925 12:26:02.753752 24087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 12:26:02.754388 23969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.754536 23969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d9dd3373b44eb8b98c9c41433c5160, State: Initialized, Role: FOLLOWER
I20260925 12:26:02.754619 23969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } }
I20260925 12:26:02.755012 23969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 12:26:02.754983 24094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.755077 24094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.756011 23942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 12:26:02.756978 23942 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [term 1 FOLLOWER]: Refusing update from remote peer 79ffa656a4204ce3bfa4e1f49870311e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 12:26:02.757211 23943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79ffa656a4204ce3bfa4e1f49870311e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "34d9dd3373b44eb8b98c9c41433c5160"
I20260925 12:26:02.757467 24084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 12:26:02.757965 23943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79ffa656a4204ce3bfa4e1f49870311e in term 1.
I20260925 12:26:02.761466 24094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ffa656a4204ce3bfa4e1f49870311e. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.761567 24094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 12:26:02.761610 24094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:02.761653 24094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 12:26:02.765362 24105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 12:26:02.766099 24105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 12:26:02.766018 24106 mvcc.cc:205] Tried to move back new op lower bound from 7333229210633523200 to 7333229210454990848. Current Snapshot: MvccSnapshot[applied={T|T < 7333229210521042944}]
W20260925 12:26:02.792904 23897 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.788291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.1.47.190:42579)
0925 12:26:02.788343 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.788345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.792103 (+ 3758us) server_negotiation.cc:407] Connection header received
0925 12:26:02.792230 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.792231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.792244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.792278 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.792818 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56494: BlockingRecv error: recv got EOF from 127.0.0.1:56494 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 12:26:02.793807 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.788576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.1.47.189:41355)
0925 12:26:02.788621 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.788623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.793158 (+ 4535us) server_negotiation.cc:407] Connection header received
0925 12:26:02.793272 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.793273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.793287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.793322 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.793740 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47672: BlockingRecv error: recv got EOF from 127.0.0.1:47672 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 12:26:02.794625 24049 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.788986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.1.47.188:35597)
0925 12:26:02.789028 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.789030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.794077 (+ 5047us) server_negotiation.cc:407] Connection header received
0925 12:26:02.794182 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.794184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.794195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.794229 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.794560 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47146: BlockingRecv error: recv got EOF from 127.0.0.1:47146 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 12:26:02.797403 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.796619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47686 (local address 127.1.47.189:41355)
0925 12:26:02.796667 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.796669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.796809 (+ 140us) server_negotiation.cc:407] Connection header received
0925 12:26:02.796931 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.796932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.796955 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.796988 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.797351 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47686: BlockingRecv error: recv got EOF from 127.0.0.1:47686 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260925 12:26:02.800980 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.800023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47702 (local address 127.1.47.189:41355)
0925 12:26:02.800063 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.800065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.800367 (+ 302us) server_negotiation.cc:407] Connection header received
0925 12:26:02.800473 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.800474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.800486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.800518 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.800930 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47702: BlockingRecv error: recv got EOF from 127.0.0.1:47702 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 12:26:02.805147 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.804234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47716 (local address 127.1.47.189:41355)
0925 12:26:02.804273 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.804275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.804593 (+ 318us) server_negotiation.cc:407] Connection header received
0925 12:26:02.804701 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.804702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.804713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.804745 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.805076 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47716: BlockingRecv error: recv got EOF from 127.0.0.1:47716 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 12:26:02.811484 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.810583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.1.47.189:41355)
0925 12:26:02.810626 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.810628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.810907 (+ 279us) server_negotiation.cc:407] Connection header received
0925 12:26:02.811018 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.811019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.811031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.811064 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.811425 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47728: BlockingRecv error: recv got EOF from 127.0.0.1:47728 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 12:26:02.822198 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.821155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.1.47.189:41355)
0925 12:26:02.821204 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.821206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.821505 (+ 299us) server_negotiation.cc:407] Connection header received
0925 12:26:02.821638 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.821640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.821654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.821687 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.822135 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47740: BlockingRecv error: recv got EOF from 127.0.0.1:47740 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 12:26:02.840699 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.839678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.1.47.189:41355)
0925 12:26:02.839726 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.839729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.840066 (+ 337us) server_negotiation.cc:407] Connection header received
0925 12:26:02.840177 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.840179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.840192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.840223 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.840635 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47748: BlockingRecv error: recv got EOF from 127.0.0.1:47748 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 12:26:02.875092 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.874072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.1.47.189:41355)
0925 12:26:02.874120 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.874122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.874423 (+ 301us) server_negotiation.cc:407] Connection header received
0925 12:26:02.874541 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.874542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.874555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.874588 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.875026 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47758: BlockingRecv error: recv got EOF from 127.0.0.1:47758 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 12:26:02.941465 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:02.940707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.1.47.189:41355)
0925 12:26:02.940799 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:02.940801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:02.940855 (+ 54us) server_negotiation.cc:407] Connection header received
0925 12:26:02.940949 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:02.940950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:02.940959 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:02.941007 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:02.941416 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47774: BlockingRecv error: recv got EOF from 127.0.0.1:47774 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260925 12:26:03.071738 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:03.070857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.1.47.189:41355)
0925 12:26:03.070956 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.070958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.071073 (+ 115us) server_negotiation.cc:407] Connection header received
0925 12:26:03.071180 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.071181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.071194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.071288 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:03.071667 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47786: BlockingRecv error: recv got EOF from 127.0.0.1:47786 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 12:26:03.226356 24084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79ffa656a4204ce3bfa4e1f49870311e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ffa656a4204ce3bfa4e1f49870311e" member_type: VOTER last_known_addr { host: "127.1.47.190" port: 42579 } } peers { permanent_uuid: "34d9dd3373b44eb8b98c9c41433c5160" member_type: VOTER last_known_addr { host: "127.1.47.189" port: 41355 } } peers { permanent_uuid: "7b120bc82c7b4a1fbc20c96932fc98de" member_type: VOTER last_known_addr { host: "127.1.47.188" port: 35597 } } }
I20260925 12:26:03.226547 24084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffa656a4204ce3bfa4e1f49870311e [sys.catalog]: This master's current role is: LEADER
W20260925 12:26:03.329704 23973 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 12:26:03.329060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.1.47.189:41355)
0925 12:26:03.329147 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.329150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.329180 (+ 30us) server_negotiation.cc:407] Connection header received
0925 12:26:03.329209 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.329210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.329225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.329300 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 12:26:03.329651 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47796: BlockingRecv error: recv got EOF from 127.0.0.1:47796 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 12:26:03.704782 24062 catalog_manager.cc:1259] Loaded cluster ID: d6505c08ef22432985f3d95d562ebdf8
I20260925 12:26:03.704859 24062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: loading cluster ID for follower catalog manager: success
I20260925 12:26:03.705757 24062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: acquiring CA information for follower catalog manager: success
I20260925 12:26:03.706079 24062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b120bc82c7b4a1fbc20c96932fc98de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:26:03.766618 24105 catalog_manager.cc:1259] Loaded cluster ID: d6505c08ef22432985f3d95d562ebdf8
I20260925 12:26:03.766728 24105 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: loading cluster ID for follower catalog manager: success
I20260925 12:26:03.767571 24105 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: acquiring CA information for follower catalog manager: success
I20260925 12:26:03.767925 24105 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34d9dd3373b44eb8b98c9c41433c5160: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 12:26:03.845840 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:03.843080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47800 (local address 127.1.47.189:41355)
0925 12:26:03.843223 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.843225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.843293 (+ 68us) server_negotiation.cc:407] Connection header received
0925 12:26:03.843354 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.843355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.843405 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.843470 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:03.843632 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.844177 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.845040 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.845263 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.845321 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:03.845480 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:03.845598 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:03.845657 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:03.845700 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 12:26:03.845716 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260925 12:26:03.851011 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:03.847673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.1.47.188:35597)
0925 12:26:03.847806 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.847807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.847828 (+ 21us) server_negotiation.cc:407] Connection header received
0925 12:26:03.847971 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.847972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.848007 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.848084 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:03.848164 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.848787 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.849773 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.850065 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.850148 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:03.850523 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:03.850690 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:03.850770 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:03.850890 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 12:26:03.850917 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260925 12:26:03.855711 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:03.852988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.1.47.188:35597)
0925 12:26:03.853032 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.853034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.853183 (+ 149us) server_negotiation.cc:407] Connection header received
0925 12:26:03.853255 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.853256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.853266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.853320 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:03.853461 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.853922 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.854917 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.855187 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.855265 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:03.855397 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:03.855435 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:03.855497 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:03.855614 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 12:26:03.855639 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 12:26:03.856218 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:03.853002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.1.47.190:42579)
0925 12:26:03.853124 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.853126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.853133 (+ 7us) server_negotiation.cc:407] Connection header received
0925 12:26:03.853177 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.853178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.853228 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.853449 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:03.853488 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.854229 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.855272 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.855599 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.855686 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:03.855890 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:03.855995 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:03.856044 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:03.856116 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 12:26:03.856141 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260925 12:26:03.856223 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 12:26:03.852761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.1.47.189:41355)
0925 12:26:03.852805 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 12:26:03.852808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 12:26:03.853055 (+ 247us) server_negotiation.cc:407] Connection header received
0925 12:26:03.853158 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 12:26:03.853159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 12:26:03.853171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 12:26:03.853199 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 12:26:03.853381 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.854123 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.855079 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 12:26:03.855376 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 12:26:03.855441 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 12:26:03.855627 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 12:26:03.855664 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 12:26:03.855711 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 12:26:03.856107 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 12:26:03.856132 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 12:26:03.862195 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23826
I20260925 12:26:03.867511 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23901
I20260925 12:26:03.872615 1214 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuGajnc/build/release/bin/kudu with pid 23976
2026-09-25T12:26:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2895 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5723 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274904 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST